2023-04-08 22:03:12,529 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-04-08 22:03:12,741 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620 2023-04-08 22:03:12,754 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-04-08 22:03:12,754 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1, deleteOnExit=true 2023-04-08 22:03:12,755 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-04-08 22:03:12,755 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/test.cache.data in system properties and HBase conf 2023-04-08 22:03:12,756 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/hadoop.tmp.dir in system properties and HBase conf 2023-04-08 22:03:12,756 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/hadoop.log.dir in system properties and HBase conf 2023-04-08 22:03:12,757 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/mapreduce.cluster.local.dir in system properties and HBase conf 2023-04-08 22:03:12,757 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-04-08 22:03:12,757 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-04-08 22:03:12,865 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-04-08 22:03:13,232 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-04-08 22:03:13,237 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-04-08 22:03:13,237 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-04-08 22:03:13,238 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-04-08 22:03:13,238 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-08 22:03:13,239 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-04-08 22:03:13,239 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-04-08 22:03:13,239 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-04-08 22:03:13,240 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-08 22:03:13,240 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-04-08 22:03:13,241 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/nfs.dump.dir in system properties and HBase conf 2023-04-08 22:03:13,241 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/java.io.tmpdir in system properties and HBase conf 2023-04-08 22:03:13,241 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/dfs.journalnode.edits.dir in system properties and HBase conf 2023-04-08 22:03:13,242 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-04-08 22:03:13,242 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-04-08 22:03:13,704 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-08 22:03:13,708 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-08 22:03:13,989 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-04-08 22:03:14,148 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-04-08 22:03:14,165 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-08 22:03:14,200 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-04-08 22:03:14,228 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/java.io.tmpdir/Jetty_localhost_localdomain_37137_hdfs____ubyzwr/webapp 2023-04-08 22:03:14,375 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:37137 2023-04-08 22:03:14,383 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-04-08 22:03:14,383 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-04-08 22:03:14,722 WARN [Listener at localhost.localdomain/34893] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-08 22:03:14,818 WARN [Listener at localhost.localdomain/34893] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-08 22:03:14,839 WARN [Listener at localhost.localdomain/34893] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-08 22:03:14,845 INFO [Listener at localhost.localdomain/34893] log.Slf4jLog(67): jetty-6.1.26 2023-04-08 22:03:14,850 INFO [Listener at localhost.localdomain/34893] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/java.io.tmpdir/Jetty_localhost_44425_datanode____.wx3vs2/webapp 2023-04-08 22:03:14,934 INFO [Listener at localhost.localdomain/34893] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:44425 2023-04-08 22:03:15,230 WARN [Listener at localhost.localdomain/42021] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-08 22:03:15,242 WARN [Listener at localhost.localdomain/42021] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-08 22:03:15,247 WARN [Listener at localhost.localdomain/42021] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-08 22:03:15,249 INFO [Listener at localhost.localdomain/42021] log.Slf4jLog(67): jetty-6.1.26 2023-04-08 22:03:15,256 INFO [Listener at localhost.localdomain/42021] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/java.io.tmpdir/Jetty_localhost_35517_datanode____uvi6lc/webapp 2023-04-08 22:03:15,366 INFO [Listener at localhost.localdomain/42021] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35517 2023-04-08 22:03:15,374 WARN [Listener at localhost.localdomain/35211] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-08 22:03:15,394 WARN [Listener at localhost.localdomain/35211] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-04-08 22:03:15,400 WARN [Listener at localhost.localdomain/35211] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-04-08 22:03:15,402 INFO [Listener at localhost.localdomain/35211] log.Slf4jLog(67): jetty-6.1.26 2023-04-08 22:03:15,409 INFO [Listener at localhost.localdomain/35211] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/java.io.tmpdir/Jetty_localhost_38071_datanode____1w70fg/webapp 2023-04-08 22:03:15,503 INFO [Listener at localhost.localdomain/35211] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38071 2023-04-08 22:03:15,515 WARN [Listener at localhost.localdomain/46567] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-04-08 22:03:15,701 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9173f55bcdd48cac: Processing first storage report for DS-b6b00b6a-9392-4857-be6e-351d3900d7eb from datanode da4aac5e-824a-49d2-8862-fa92d980a6ea 2023-04-08 22:03:15,703 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9173f55bcdd48cac: from storage DS-b6b00b6a-9392-4857-be6e-351d3900d7eb node DatanodeRegistration(127.0.0.1:40741, datanodeUuid=da4aac5e-824a-49d2-8862-fa92d980a6ea, infoPort=38461, infoSecurePort=0, ipcPort=42021, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-04-08 22:03:15,703 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x32e496cdcf6e5647: Processing first storage report for DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f from datanode 0f83e86d-490c-4535-bc1b-a2692e207dbe 2023-04-08 22:03:15,704 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x32e496cdcf6e5647: from storage DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f node DatanodeRegistration(127.0.0.1:41459, datanodeUuid=0f83e86d-490c-4535-bc1b-a2692e207dbe, infoPort=45091, infoSecurePort=0, ipcPort=46567, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-08 22:03:15,704 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe1a5ed2a733cb057: Processing first storage report for DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2 from datanode 77a94139-75e1-486c-a582-be7e9aa2826d 2023-04-08 22:03:15,704 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe1a5ed2a733cb057: from storage DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2 node DatanodeRegistration(127.0.0.1:37427, datanodeUuid=77a94139-75e1-486c-a582-be7e9aa2826d, infoPort=45579, infoSecurePort=0, ipcPort=35211, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-04-08 22:03:15,704 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9173f55bcdd48cac: Processing first storage report for DS-ccdd8dc2-cc56-4592-aab8-d4bbf31de0fd from datanode da4aac5e-824a-49d2-8862-fa92d980a6ea 2023-04-08 22:03:15,705 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9173f55bcdd48cac: from storage DS-ccdd8dc2-cc56-4592-aab8-d4bbf31de0fd node DatanodeRegistration(127.0.0.1:40741, datanodeUuid=da4aac5e-824a-49d2-8862-fa92d980a6ea, infoPort=38461, infoSecurePort=0, ipcPort=42021, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-08 22:03:15,705 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x32e496cdcf6e5647: Processing first storage report for DS-62208575-f1b9-45d7-b033-6e4f43542a37 from datanode 0f83e86d-490c-4535-bc1b-a2692e207dbe 2023-04-08 22:03:15,705 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x32e496cdcf6e5647: from storage DS-62208575-f1b9-45d7-b033-6e4f43542a37 node DatanodeRegistration(127.0.0.1:41459, datanodeUuid=0f83e86d-490c-4535-bc1b-a2692e207dbe, infoPort=45091, infoSecurePort=0, ipcPort=46567, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-08 22:03:15,705 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xe1a5ed2a733cb057: Processing first storage report for DS-0d4e4109-43f9-4d19-912d-84f02bfb02b7 from datanode 77a94139-75e1-486c-a582-be7e9aa2826d 2023-04-08 22:03:15,705 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xe1a5ed2a733cb057: from storage DS-0d4e4109-43f9-4d19-912d-84f02bfb02b7 node DatanodeRegistration(127.0.0.1:37427, datanodeUuid=77a94139-75e1-486c-a582-be7e9aa2826d, infoPort=45579, infoSecurePort=0, ipcPort=35211, storageInfo=lv=-57;cid=testClusterID;nsid=269398097;c=1680991393780), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-04-08 22:03:16,069 DEBUG [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620 2023-04-08 22:03:16,121 INFO [Listener at localhost.localdomain/46567] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/zookeeper_0, clientPort=55350, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-04-08 22:03:16,134 INFO [Listener at localhost.localdomain/46567] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55350 2023-04-08 22:03:16,140 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:16,142 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:16,779 INFO [Listener at localhost.localdomain/46567] util.FSUtils(471): Created version file at hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7 with version=8 2023-04-08 22:03:16,779 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/hbase-staging 2023-04-08 22:03:17,062 INFO [Listener at localhost.localdomain/46567] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-04-08 22:03:17,456 INFO [Listener at localhost.localdomain/46567] client.ConnectionUtils(127): master/jenkins-hbase20:0 server-side Connection retries=45 2023-04-08 22:03:17,481 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,482 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,482 INFO [Listener at localhost.localdomain/46567] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-08 22:03:17,482 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,483 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-08 22:03:17,604 INFO [Listener at localhost.localdomain/46567] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-04-08 22:03:17,667 DEBUG [Listener at localhost.localdomain/46567] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-04-08 22:03:17,748 INFO [Listener at localhost.localdomain/46567] ipc.NettyRpcServer(120): Bind to /148.251.75.209:39273 2023-04-08 22:03:17,758 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,759 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,779 INFO [Listener at localhost.localdomain/46567] zookeeper.RecoverableZooKeeper(93): Process identifier=master:39273 connecting to ZooKeeper ensemble=127.0.0.1:55350 2023-04-08 22:03:17,814 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:392730x0, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-08 22:03:17,816 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:39273-0x100929b39380000 connected 2023-04-08 22:03:17,839 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-08 22:03:17,840 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:03:17,844 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-08 22:03:17,851 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=39273 2023-04-08 22:03:17,852 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=39273 2023-04-08 22:03:17,854 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=39273 2023-04-08 22:03:17,854 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=39273 2023-04-08 22:03:17,855 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=39273 2023-04-08 22:03:17,860 INFO [Listener at localhost.localdomain/46567] master.HMaster(444): hbase.rootdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7, hbase.cluster.distributed=false 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,922 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-08 22:03:17,926 INFO [Listener at localhost.localdomain/46567] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-08 22:03:17,929 INFO [Listener at localhost.localdomain/46567] ipc.NettyRpcServer(120): Bind to /148.251.75.209:43551 2023-04-08 22:03:17,931 INFO [Listener at localhost.localdomain/46567] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-08 22:03:17,937 DEBUG [Listener at localhost.localdomain/46567] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-08 22:03:17,938 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,940 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,942 INFO [Listener at localhost.localdomain/46567] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43551 connecting to ZooKeeper ensemble=127.0.0.1:55350 2023-04-08 22:03:17,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:435510x0, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-08 22:03:17,947 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:435510x0, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-08 22:03:17,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43551-0x100929b39380001 connected 2023-04-08 22:03:17,948 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:03:17,949 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-08 22:03:17,954 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43551 2023-04-08 22:03:17,955 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43551 2023-04-08 22:03:17,955 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43551 2023-04-08 22:03:17,955 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43551 2023-04-08 22:03:17,956 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43551 2023-04-08 22:03:17,967 INFO [Listener at localhost.localdomain/46567] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-04-08 22:03:17,968 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,968 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,968 INFO [Listener at localhost.localdomain/46567] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-08 22:03:17,969 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:17,969 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-08 22:03:17,969 INFO [Listener at localhost.localdomain/46567] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-08 22:03:17,971 INFO [Listener at localhost.localdomain/46567] ipc.NettyRpcServer(120): Bind to /148.251.75.209:41749 2023-04-08 22:03:17,971 INFO [Listener at localhost.localdomain/46567] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-08 22:03:17,972 DEBUG [Listener at localhost.localdomain/46567] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-08 22:03:17,973 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,976 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:17,979 INFO [Listener at localhost.localdomain/46567] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41749 connecting to ZooKeeper ensemble=127.0.0.1:55350 2023-04-08 22:03:17,984 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:417490x0, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-08 22:03:17,985 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:417490x0, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-08 22:03:17,985 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41749-0x100929b39380002 connected 2023-04-08 22:03:17,986 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:03:17,986 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-08 22:03:17,990 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41749 2023-04-08 22:03:17,990 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41749 2023-04-08 22:03:17,991 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41749 2023-04-08 22:03:17,991 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41749 2023-04-08 22:03:17,991 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41749 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] client.ConnectionUtils(127): regionserver/jenkins-hbase20:0 server-side Connection retries=45 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-04-08 22:03:18,003 INFO [Listener at localhost.localdomain/46567] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-04-08 22:03:18,005 INFO [Listener at localhost.localdomain/46567] ipc.NettyRpcServer(120): Bind to /148.251.75.209:41779 2023-04-08 22:03:18,005 INFO [Listener at localhost.localdomain/46567] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-04-08 22:03:18,006 DEBUG [Listener at localhost.localdomain/46567] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-04-08 22:03:18,007 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:18,009 INFO [Listener at localhost.localdomain/46567] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:18,011 INFO [Listener at localhost.localdomain/46567] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41779 connecting to ZooKeeper ensemble=127.0.0.1:55350 2023-04-08 22:03:18,015 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:417790x0, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-04-08 22:03:18,016 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:417790x0, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-08 22:03:18,016 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:41779-0x100929b39380003 connected 2023-04-08 22:03:18,017 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:03:18,017 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-04-08 22:03:18,020 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41779 2023-04-08 22:03:18,020 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41779 2023-04-08 22:03:18,020 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41779 2023-04-08 22:03:18,021 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41779 2023-04-08 22:03:18,022 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41779 2023-04-08 22:03:18,024 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,032 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-08 22:03:18,033 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,050 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-08 22:03:18,050 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-08 22:03:18,050 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-08 22:03:18,050 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-04-08 22:03:18,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,051 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-08 22:03:18,052 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase20.apache.org,39273,1680991396909 from backup master directory 2023-04-08 22:03:18,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-04-08 22:03:18,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-04-08 22:03:18,057 WARN [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-08 22:03:18,057 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,060 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-04-08 22:03:18,061 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-04-08 22:03:18,154 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/hbase.id with ID: f6fe47ad-863e-439d-8e52-03cd741ab806 2023-04-08 22:03:18,202 INFO [master/jenkins-hbase20:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-04-08 22:03:18,219 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,269 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x38785559 to 127.0.0.1:55350 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-08 22:03:18,298 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4397cfe6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-08 22:03:18,319 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-08 22:03:18,321 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-04-08 22:03:18,335 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-04-08 22:03:18,335 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-04-08 22:03:18,337 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-08 22:03:18,340 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-04-08 22:03:18,342 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-08 22:03:18,371 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store-tmp 2023-04-08 22:03:18,446 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:18,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-08 22:03:18,447 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:03:18,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:03:18,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-08 22:03:18,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:03:18,447 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:03:18,447 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-08 22:03:18,449 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/WALs/jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,470 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C39273%2C1680991396909, suffix=, logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/WALs/jenkins-hbase20.apache.org,39273,1680991396909, archiveDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/oldWALs, maxLogs=10 2023-04-08 22:03:18,524 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK] 2023-04-08 22:03:18,524 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK] 2023-04-08 22:03:18,524 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK] 2023-04-08 22:03:18,534 DEBUG [RS-EventLoopGroup-5-3] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-04-08 22:03:18,603 INFO [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/WALs/jenkins-hbase20.apache.org,39273,1680991396909/jenkins-hbase20.apache.org%2C39273%2C1680991396909.1680991398479 2023-04-08 22:03:18,604 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK], DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK], DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK]] 2023-04-08 22:03:18,604 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:18,605 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:18,608 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,609 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,670 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,678 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-04-08 22:03:18,701 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-04-08 22:03:18,712 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:18,719 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,721 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,736 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-04-08 22:03:18,740 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:18,742 INFO [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67566132, jitterRate=0.00681382417678833}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-08 22:03:18,742 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-08 22:03:18,743 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-04-08 22:03:18,762 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:18,762 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:18,764 INFO [master/jenkins-hbase20:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:18,766 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:03:18,795 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 28 msec 2023-04-08 22:03:18,795 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-04-08 22:03:18,818 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-04-08 22:03:18,824 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-04-08 22:03:18,847 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-04-08 22:03:18,850 INFO [master/jenkins-hbase20:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-04-08 22:03:18,851 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-04-08 22:03:18,856 INFO [master/jenkins-hbase20:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-04-08 22:03:18,860 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-04-08 22:03:18,862 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,864 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-04-08 22:03:18,864 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-04-08 22:03:18,879 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-04-08 22:03:18,885 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-08 22:03:18,885 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-08 22:03:18,885 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-08 22:03:18,885 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-04-08 22:03:18,885 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,886 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase20.apache.org,39273,1680991396909, sessionid=0x100929b39380000, setting cluster-up flag (Was=false) 2023-04-08 22:03:18,901 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,905 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-04-08 22:03:18,906 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,910 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:18,914 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-04-08 22:03:18,915 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:18,918 WARN [master/jenkins-hbase20:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.hbase-snapshot/.tmp 2023-04-08 22:03:18,926 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(952): ClusterId : f6fe47ad-863e-439d-8e52-03cd741ab806 2023-04-08 22:03:18,926 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(952): ClusterId : f6fe47ad-863e-439d-8e52-03cd741ab806 2023-04-08 22:03:18,926 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(952): ClusterId : f6fe47ad-863e-439d-8e52-03cd741ab806 2023-04-08 22:03:18,932 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-08 22:03:18,932 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-08 22:03:18,932 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-04-08 22:03:18,938 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-08 22:03:18,938 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-08 22:03:18,938 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-04-08 22:03:18,938 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-08 22:03:18,938 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-08 22:03:18,938 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-04-08 22:03:18,941 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-08 22:03:18,941 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-08 22:03:18,944 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ReadOnlyZKClient(139): Connect 0x28fb1460 to 127.0.0.1:55350 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-08 22:03:18,944 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ReadOnlyZKClient(139): Connect 0x3c7f3513 to 127.0.0.1:55350 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-08 22:03:18,944 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-04-08 22:03:18,947 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ReadOnlyZKClient(139): Connect 0x23e3cc80 to 127.0.0.1:55350 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-08 22:03:18,956 DEBUG [RS:0;jenkins-hbase20:43551] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32b8ccec, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-08 22:03:18,956 DEBUG [RS:2;jenkins-hbase20:41779] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@263cff0f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-08 22:03:18,956 DEBUG [RS:0;jenkins-hbase20:43551] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36875e63, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-04-08 22:03:18,956 DEBUG [RS:2;jenkins-hbase20:41779] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@22037f10, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-04-08 22:03:18,958 DEBUG [RS:1;jenkins-hbase20:41749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d24268c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-08 22:03:18,958 DEBUG [RS:1;jenkins-hbase20:41749] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@291ba7f0, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-04-08 22:03:18,977 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase20:43551 2023-04-08 22:03:18,980 INFO [RS:0;jenkins-hbase20:43551] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-08 22:03:18,981 INFO [RS:0;jenkins-hbase20:43551] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-08 22:03:18,981 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1023): About to register with Master. 2023-04-08 22:03:18,981 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase20:41779 2023-04-08 22:03:18,981 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase20:41749 2023-04-08 22:03:18,981 INFO [RS:2;jenkins-hbase20:41779] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-08 22:03:18,981 INFO [RS:1;jenkins-hbase20:41749] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-04-08 22:03:18,983 INFO [RS:1;jenkins-hbase20:41749] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-08 22:03:18,983 INFO [RS:2;jenkins-hbase20:41779] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-04-08 22:03:18,983 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1023): About to register with Master. 2023-04-08 22:03:18,983 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1023): About to register with Master. 2023-04-08 22:03:18,985 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:41749, startcode=1680991397967 2023-04-08 22:03:18,985 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:41779, startcode=1680991398002 2023-04-08 22:03:18,985 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:43551, startcode=1680991397921 2023-04-08 22:03:19,003 DEBUG [RS:1;jenkins-hbase20:41749] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-08 22:03:19,003 DEBUG [RS:0;jenkins-hbase20:43551] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-08 22:03:19,003 DEBUG [RS:2;jenkins-hbase20:41779] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-04-08 22:03:19,060 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:19,064 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:41597, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-04-08 22:03:19,064 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:60747, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-04-08 22:03:19,064 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:59315, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-04-08 22:03:19,070 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-04-08 22:03:19,070 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-04-08 22:03:19,070 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-04-08 22:03:19,070 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=5, maxPoolSize=5 2023-04-08 22:03:19,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase20:0, corePoolSize=10, maxPoolSize=10 2023-04-08 22:03:19,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-04-08 22:03:19,071 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39273] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-04-08 22:03:19,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-04-08 22:03:19,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:577) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-04-08 22:03:19,076 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991429076 2023-04-08 22:03:19,079 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-04-08 22:03:19,083 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:19,084 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-04-08 22:03:19,091 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-08 22:03:19,093 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-04-08 22:03:19,102 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-04-08 22:03:19,102 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-04-08 22:03:19,103 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-04-08 22:03:19,103 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-04-08 22:03:19,108 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,110 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-04-08 22:03:19,112 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(2829): Master is not running yet 2023-04-08 22:03:19,112 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-04-08 22:03:19,113 WARN [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-04-08 22:03:19,112 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(2829): Master is not running yet 2023-04-08 22:03:19,112 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(2829): Master is not running yet 2023-04-08 22:03:19,113 WARN [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-04-08 22:03:19,114 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-04-08 22:03:19,113 WARN [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1031): reportForDuty failed; sleeping 100 ms and then retrying. 2023-04-08 22:03:19,117 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-04-08 22:03:19,117 INFO [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-04-08 22:03:19,122 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1680991399121,5,FailOnTimeoutGroup] 2023-04-08 22:03:19,123 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1680991399122,5,FailOnTimeoutGroup] 2023-04-08 22:03:19,123 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,123 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-04-08 22:03:19,126 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,128 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,143 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:19,144 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:19,144 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7 2023-04-08 22:03:19,171 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:19,173 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-08 22:03:19,176 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/info 2023-04-08 22:03:19,177 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-08 22:03:19,177 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,178 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-08 22:03:19,180 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/rep_barrier 2023-04-08 22:03:19,181 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-08 22:03:19,181 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,182 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-08 22:03:19,184 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/table 2023-04-08 22:03:19,184 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-08 22:03:19,185 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,187 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740 2023-04-08 22:03:19,188 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740 2023-04-08 22:03:19,191 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-08 22:03:19,193 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-08 22:03:19,197 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:19,198 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68692631, jitterRate=0.02359996736049652}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-08 22:03:19,199 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-08 22:03:19,199 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-08 22:03:19,199 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-08 22:03:19,199 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-08 22:03:19,199 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-08 22:03:19,199 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-08 22:03:19,201 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-08 22:03:19,201 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-08 22:03:19,207 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:19,207 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-04-08 22:03:19,214 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-04-08 22:03:19,215 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:43551, startcode=1680991397921 2023-04-08 22:03:19,215 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:41749, startcode=1680991397967 2023-04-08 22:03:19,215 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase20.apache.org,39273,1680991396909 with isa=jenkins-hbase20.apache.org/148.251.75.209:41779, startcode=1680991398002 2023-04-08 22:03:19,221 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,221 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,222 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=39273] master.ServerManager(394): Registering regionserver=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,228 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7 2023-04-08 22:03:19,228 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7 2023-04-08 22:03:19,229 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34893 2023-04-08 22:03:19,229 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34893 2023-04-08 22:03:19,229 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-08 22:03:19,229 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-08 22:03:19,229 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7 2023-04-08 22:03:19,230 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:34893 2023-04-08 22:03:19,230 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-04-08 22:03:19,233 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-04-08 22:03:19,235 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:03:19,237 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ZKUtil(162): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,237 WARN [RS:2;jenkins-hbase20:41779] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-08 22:03:19,237 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ZKUtil(162): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,237 INFO [RS:2;jenkins-hbase20:41779] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-08 22:03:19,237 WARN [RS:1;jenkins-hbase20:41749] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-08 22:03:19,238 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,238 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ZKUtil(162): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,238 INFO [RS:1;jenkins-hbase20:41749] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-08 22:03:19,238 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,239 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,41779,1680991398002] 2023-04-08 22:03:19,239 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,41749,1680991397967] 2023-04-08 22:03:19,239 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase20.apache.org,43551,1680991397921] 2023-04-08 22:03:19,239 WARN [RS:0;jenkins-hbase20:43551] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-04-08 22:03:19,239 INFO [RS:0;jenkins-hbase20:43551] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-08 22:03:19,239 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,244 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-08 22:03:19,254 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ZKUtil(162): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,254 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ZKUtil(162): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,254 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ZKUtil(162): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,254 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ZKUtil(162): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,254 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ZKUtil(162): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,255 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ZKUtil(162): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,255 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ZKUtil(162): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,255 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ZKUtil(162): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,255 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ZKUtil(162): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,264 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-08 22:03:19,264 DEBUG [RS:2;jenkins-hbase20:41779] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-08 22:03:19,264 DEBUG [RS:0;jenkins-hbase20:43551] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-04-08 22:03:19,273 INFO [RS:2;jenkins-hbase20:41779] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-08 22:03:19,273 INFO [RS:0;jenkins-hbase20:43551] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-08 22:03:19,273 INFO [RS:1;jenkins-hbase20:41749] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-04-08 22:03:19,294 INFO [RS:1;jenkins-hbase20:41749] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-08 22:03:19,294 INFO [RS:2;jenkins-hbase20:41779] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-08 22:03:19,294 INFO [RS:0;jenkins-hbase20:43551] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-04-08 22:03:19,298 INFO [RS:2;jenkins-hbase20:41779] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-08 22:03:19,298 INFO [RS:0;jenkins-hbase20:43551] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-08 22:03:19,299 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,298 INFO [RS:1;jenkins-hbase20:41749] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-04-08 22:03:19,299 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,299 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,300 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-08 22:03:19,300 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-08 22:03:19,300 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-04-08 22:03:19,308 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,308 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,308 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,309 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,309 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,310 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:2;jenkins-hbase20:41779] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:0;jenkins-hbase20:43551] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase20:0, corePoolSize=2, maxPoolSize=2 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,311 DEBUG [RS:1;jenkins-hbase20:41749] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase20:0, corePoolSize=1, maxPoolSize=1 2023-04-08 22:03:19,317 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,317 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,332 INFO [RS:1;jenkins-hbase20:41749] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-08 22:03:19,332 INFO [RS:0;jenkins-hbase20:43551] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-08 22:03:19,332 INFO [RS:2;jenkins-hbase20:41779] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-04-08 22:03:19,335 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41779,1680991398002-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,335 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,43551,1680991397921-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,335 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,41749,1680991397967-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,362 INFO [RS:1;jenkins-hbase20:41749] regionserver.Replication(203): jenkins-hbase20.apache.org,41749,1680991397967 started 2023-04-08 22:03:19,362 INFO [RS:2;jenkins-hbase20:41779] regionserver.Replication(203): jenkins-hbase20.apache.org,41779,1680991398002 started 2023-04-08 22:03:19,362 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,41749,1680991397967, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:41749, sessionid=0x100929b39380002 2023-04-08 22:03:19,362 INFO [RS:0;jenkins-hbase20:43551] regionserver.Replication(203): jenkins-hbase20.apache.org,43551,1680991397921 started 2023-04-08 22:03:19,362 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,41779,1680991398002, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:41779, sessionid=0x100929b39380003 2023-04-08 22:03:19,362 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1638): Serving as jenkins-hbase20.apache.org,43551,1680991397921, RpcServer on jenkins-hbase20.apache.org/148.251.75.209:43551, sessionid=0x100929b39380001 2023-04-08 22:03:19,362 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-08 22:03:19,362 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-08 22:03:19,362 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-04-08 22:03:19,363 DEBUG [RS:1;jenkins-hbase20:41749] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,362 DEBUG [RS:2;jenkins-hbase20:41779] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,362 DEBUG [RS:0;jenkins-hbase20:43551] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,363 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,43551,1680991397921' 2023-04-08 22:03:19,363 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41779,1680991398002' 2023-04-08 22:03:19,364 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:19,363 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41749,1680991397967' 2023-04-08 22:03:19,364 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:19,364 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:19,364 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:19,364 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:19,364 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:19,365 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-08 22:03:19,365 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-08 22:03:19,365 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-08 22:03:19,365 DEBUG [RS:1;jenkins-hbase20:41749] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,365 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-04-08 22:03:19,365 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41749,1680991397967' 2023-04-08 22:03:19,365 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-08 22:03:19,365 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-08 22:03:19,365 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-04-08 22:03:19,365 DEBUG [RS:2;jenkins-hbase20:41779] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:19,366 DEBUG [RS:0;jenkins-hbase20:43551] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:19,366 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,43551,1680991397921' 2023-04-08 22:03:19,366 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-08 22:03:19,366 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase20.apache.org,41779,1680991398002' 2023-04-08 22:03:19,366 DEBUG [RS:1;jenkins-hbase20:41749] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-08 22:03:19,366 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-04-08 22:03:19,367 DEBUG [RS:0;jenkins-hbase20:43551] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-08 22:03:19,367 DEBUG [RS:1;jenkins-hbase20:41749] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-08 22:03:19,367 DEBUG [RS:2;jenkins-hbase20:41779] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-04-08 22:03:19,367 INFO [RS:1;jenkins-hbase20:41749] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-08 22:03:19,367 INFO [RS:1;jenkins-hbase20:41749] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-08 22:03:19,367 DEBUG [RS:0;jenkins-hbase20:43551] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-08 22:03:19,367 INFO [RS:0;jenkins-hbase20:43551] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-08 22:03:19,369 INFO [RS:0;jenkins-hbase20:43551] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-08 22:03:19,369 DEBUG [RS:2;jenkins-hbase20:41779] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-04-08 22:03:19,369 INFO [RS:2;jenkins-hbase20:41779] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-04-08 22:03:19,369 INFO [RS:2;jenkins-hbase20:41779] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-04-08 22:03:19,397 DEBUG [jenkins-hbase20:39273] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-04-08 22:03:19,400 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:19,406 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:19,406 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:19,406 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:19,406 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:19,409 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,41749,1680991397967, state=OPENING 2023-04-08 22:03:19,414 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-04-08 22:03:19,415 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:19,416 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-08 22:03:19,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:19,477 INFO [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C41749%2C1680991397967, suffix=, logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41749,1680991397967, archiveDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs, maxLogs=32 2023-04-08 22:03:19,477 INFO [RS:0;jenkins-hbase20:43551] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C43551%2C1680991397921, suffix=, logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,43551,1680991397921, archiveDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs, maxLogs=32 2023-04-08 22:03:19,477 INFO [RS:2;jenkins-hbase20:41779] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C41779%2C1680991398002, suffix=, logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41779,1680991398002, archiveDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs, maxLogs=32 2023-04-08 22:03:19,501 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK] 2023-04-08 22:03:19,501 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK] 2023-04-08 22:03:19,501 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK] 2023-04-08 22:03:19,502 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK] 2023-04-08 22:03:19,505 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK] 2023-04-08 22:03:19,505 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK] 2023-04-08 22:03:19,505 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK] 2023-04-08 22:03:19,513 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK] 2023-04-08 22:03:19,513 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK] 2023-04-08 22:03:19,522 INFO [RS:2;jenkins-hbase20:41779] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41779,1680991398002/jenkins-hbase20.apache.org%2C41779%2C1680991398002.1680991399481 2023-04-08 22:03:19,522 INFO [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41749,1680991397967/jenkins-hbase20.apache.org%2C41749%2C1680991397967.1680991399481 2023-04-08 22:03:19,522 INFO [RS:0;jenkins-hbase20:43551] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,43551,1680991397921/jenkins-hbase20.apache.org%2C43551%2C1680991397921.1680991399481 2023-04-08 22:03:19,522 DEBUG [RS:2;jenkins-hbase20:41779] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK], DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK], DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK]] 2023-04-08 22:03:19,523 DEBUG [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK], DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK], DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK]] 2023-04-08 22:03:19,526 DEBUG [RS:0;jenkins-hbase20:43551] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK], DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK], DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK]] 2023-04-08 22:03:19,598 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:19,600 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-08 22:03:19,603 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:38520, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-08 22:03:19,615 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-04-08 22:03:19,616 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-04-08 22:03:19,620 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase20.apache.org%2C41749%2C1680991397967.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41749,1680991397967, archiveDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs, maxLogs=32 2023-04-08 22:03:19,640 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK] 2023-04-08 22:03:19,641 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK] 2023-04-08 22:03:19,642 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK] 2023-04-08 22:03:19,650 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/WALs/jenkins-hbase20.apache.org,41749,1680991397967/jenkins-hbase20.apache.org%2C41749%2C1680991397967.meta.1680991399622.meta 2023-04-08 22:03:19,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:40741,DS-b6b00b6a-9392-4857-be6e-351d3900d7eb,DISK], DatanodeInfoWithStorage[127.0.0.1:41459,DS-2e6f41ae-eb4c-4d9d-8c79-ad6f048cf06f,DISK], DatanodeInfoWithStorage[127.0.0.1:37427,DS-a47cfc95-b21b-457d-8566-7d8d61a73bc2,DISK]] 2023-04-08 22:03:19,651 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:19,656 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:19,657 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-04-08 22:03:19,676 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-04-08 22:03:19,679 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-04-08 22:03:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-04-08 22:03:19,684 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:19,685 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-04-08 22:03:19,685 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-04-08 22:03:19,688 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-04-08 22:03:19,690 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/info 2023-04-08 22:03:19,690 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/info 2023-04-08 22:03:19,691 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-04-08 22:03:19,692 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,692 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-04-08 22:03:19,694 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/rep_barrier 2023-04-08 22:03:19,694 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/rep_barrier 2023-04-08 22:03:19,694 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-04-08 22:03:19,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,696 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-04-08 22:03:19,697 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/table 2023-04-08 22:03:19,698 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/table 2023-04-08 22:03:19,698 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-04-08 22:03:19,699 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:19,702 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740 2023-04-08 22:03:19,705 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740 2023-04-08 22:03:19,709 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-04-08 22:03:19,712 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-04-08 22:03:19,713 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64226269, jitterRate=-0.042954012751579285}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-04-08 22:03:19,714 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-04-08 22:03:19,725 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1680991399591 2023-04-08 22:03:19,743 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-04-08 22:03:19,744 INFO [RS_OPEN_META-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-04-08 22:03:19,744 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase20.apache.org,41749,1680991397967, state=OPEN 2023-04-08 22:03:19,747 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-04-08 22:03:19,747 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-04-08 22:03:19,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-04-08 22:03:19,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase20.apache.org,41749,1680991397967 in 328 msec 2023-04-08 22:03:19,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-04-08 22:03:19,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 544 msec 2023-04-08 22:03:19,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 789 msec 2023-04-08 22:03:19,771 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1680991399771, completionTime=-1 2023-04-08 22:03:19,772 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-04-08 22:03:19,772 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:19,825 DEBUG [hconnection-0x3a7752df-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-08 22:03:19,829 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:38530, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-08 22:03:19,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:19,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991459845 2023-04-08 22:03:19,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991519845 2023-04-08 22:03:19,845 INFO [master/jenkins-hbase20:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 73 msec 2023-04-08 22:03:19,878 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39273,1680991396909-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,878 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39273,1680991396909-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,879 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39273,1680991396909-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,880 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase20:39273, period=300000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,880 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-04-08 22:03:19,887 DEBUG [master/jenkins-hbase20:0.Chore.1] janitor.CatalogJanitor(175): 2023-04-08 22:03:19,894 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-04-08 22:03:19,895 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-04-08 22:03:19,904 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:19,907 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:19,910 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:19,930 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:19,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e empty. 2023-04-08 22:03:19,933 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:19,933 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-04-08 22:03:19,974 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:19,977 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => b6167fccb8f7fec5f3eb188ccc6fd56e, NAME => 'hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing b6167fccb8f7fec5f3eb188ccc6fd56e, disabling compactions & flushes 2023-04-08 22:03:19,997 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. after waiting 0 ms 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:19,997 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:19,997 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for b6167fccb8f7fec5f3eb188ccc6fd56e: 2023-04-08 22:03:20,002 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:20,016 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680991400004"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991400004"}]},"ts":"1680991400004"} 2023-04-08 22:03:20,041 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:20,043 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:20,047 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991400043"}]},"ts":"1680991400043"} 2023-04-08 22:03:20,053 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-04-08 22:03:20,056 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:20,058 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:20,058 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:20,058 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:20,058 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:20,061 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b6167fccb8f7fec5f3eb188ccc6fd56e, ASSIGN}] 2023-04-08 22:03:20,066 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=b6167fccb8f7fec5f3eb188ccc6fd56e, ASSIGN 2023-04-08 22:03:20,071 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=b6167fccb8f7fec5f3eb188ccc6fd56e, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:20,223 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:20,224 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:20,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680991400224"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991400224"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991400224"}]},"ts":"1680991400224"} 2023-04-08 22:03:20,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure b6167fccb8f7fec5f3eb188ccc6fd56e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:20,392 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:20,392 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b6167fccb8f7fec5f3eb188ccc6fd56e, NAME => 'hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:20,393 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:20,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:20,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,394 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,396 INFO [StoreOpener-b6167fccb8f7fec5f3eb188ccc6fd56e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,398 DEBUG [StoreOpener-b6167fccb8f7fec5f3eb188ccc6fd56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/info 2023-04-08 22:03:20,399 DEBUG [StoreOpener-b6167fccb8f7fec5f3eb188ccc6fd56e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/info 2023-04-08 22:03:20,399 INFO [StoreOpener-b6167fccb8f7fec5f3eb188ccc6fd56e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b6167fccb8f7fec5f3eb188ccc6fd56e columnFamilyName info 2023-04-08 22:03:20,400 INFO [StoreOpener-b6167fccb8f7fec5f3eb188ccc6fd56e-1] regionserver.HStore(310): Store=b6167fccb8f7fec5f3eb188ccc6fd56e/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:20,402 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,404 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,408 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:03:20,411 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:20,412 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened b6167fccb8f7fec5f3eb188ccc6fd56e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73400431, jitterRate=0.09375165402889252}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-08 22:03:20,412 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for b6167fccb8f7fec5f3eb188ccc6fd56e: 2023-04-08 22:03:20,414 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e., pid=6, masterSystemTime=1680991400386 2023-04-08 22:03:20,418 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:20,418 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:03:20,419 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:20,420 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1680991400419"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991400419"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991400419"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991400419"}]},"ts":"1680991400419"} 2023-04-08 22:03:20,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-04-08 22:03:20,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure b6167fccb8f7fec5f3eb188ccc6fd56e, server=jenkins-hbase20.apache.org,41749,1680991397967 in 192 msec 2023-04-08 22:03:20,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-04-08 22:03:20,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=b6167fccb8f7fec5f3eb188ccc6fd56e, ASSIGN in 366 msec 2023-04-08 22:03:20,432 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:20,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991400432"}]},"ts":"1680991400432"} 2023-04-08 22:03:20,435 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-04-08 22:03:20,438 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:20,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 542 msec 2023-04-08 22:03:20,507 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-04-08 22:03:20,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-04-08 22:03:20,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:20,542 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:20,561 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-08 22:03:20,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 31 msec 2023-04-08 22:03:20,577 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:20,590 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-04-08 22:03:20,595 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 18 msec 2023-04-08 22:03:20,610 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-04-08 22:03:20,611 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-04-08 22:03:20,612 INFO [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.554sec 2023-04-08 22:03:20,614 INFO [master/jenkins-hbase20:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-04-08 22:03:20,615 INFO [master/jenkins-hbase20:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-04-08 22:03:20,615 INFO [master/jenkins-hbase20:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-04-08 22:03:20,617 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39273,1680991396909-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-04-08 22:03:20,617 INFO [master/jenkins-hbase20:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase20.apache.org,39273,1680991396909-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-04-08 22:03:20,625 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-04-08 22:03:20,632 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ReadOnlyZKClient(139): Connect 0x1afd0182 to 127.0.0.1:55350 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-04-08 22:03:20,636 DEBUG [Listener at localhost.localdomain/46567] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2dbedad7, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-04-08 22:03:20,650 DEBUG [hconnection-0x49d8264d-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-08 22:03:20,661 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:38532, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-08 22:03:20,669 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:03:20,697 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=691, MaxFileDescriptor=60000, SystemLoadAverage=224, ProcessCount=165, AvailableMemoryMB=4414 2023-04-08 22:03:20,701 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:20,709 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-04-08 22:03:20,713 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:50868, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-04-08 22:03:20,725 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:20,725 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:03:20,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:20,731 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:20,734 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:20,736 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:20,739 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:20,740 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 empty. 2023-04-08 22:03:20,743 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:20,743 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-04-08 22:03:20,765 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:20,767 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, NAME => 'testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:20,788 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:20,788 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, disabling compactions & flushes 2023-04-08 22:03:20,789 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:20,789 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:20,789 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. after waiting 0 ms 2023-04-08 22:03:20,789 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:20,789 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:20,789 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:20,793 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:20,795 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991400795"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991400795"}]},"ts":"1680991400795"} 2023-04-08 22:03:20,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:20,800 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:20,800 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991400800"}]},"ts":"1680991400800"} 2023-04-08 22:03:20,803 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-04-08 22:03:20,806 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:20,808 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:20,808 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:20,808 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:20,808 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:20,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, ASSIGN}] 2023-04-08 22:03:20,812 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, ASSIGN 2023-04-08 22:03:20,813 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:20,964 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:20,965 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:20,965 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991400965"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991400965"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991400965"}]},"ts":"1680991400965"} 2023-04-08 22:03:20,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:21,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, NAME => 'testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:21,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:21,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,130 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:21,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,133 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,135 DEBUG [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1 2023-04-08 22:03:21,135 DEBUG [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1 2023-04-08 22:03:21,136 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 columnFamilyName cf1 2023-04-08 22:03:21,137 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] regionserver.HStore(310): Store=9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:21,137 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,139 DEBUG [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2 2023-04-08 22:03:21,140 DEBUG [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2 2023-04-08 22:03:21,140 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 columnFamilyName cf2 2023-04-08 22:03:21,141 INFO [StoreOpener-9ad6c05dacbf0ddb8d22dd1a8d21f5f2-1] regionserver.HStore(310): Store=9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:21,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,145 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,148 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:21,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:21,154 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:21,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 9ad6c05dacbf0ddb8d22dd1a8d21f5f2; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69149795, jitterRate=0.03041224181652069}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:21,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:21,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2., pid=11, masterSystemTime=1680991401122 2023-04-08 22:03:21,159 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,160 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,160 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,161 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991401160"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991401160"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991401160"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991401160"}]},"ts":"1680991401160"} 2023-04-08 22:03:21,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-04-08 22:03:21,167 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, server=jenkins-hbase20.apache.org,41749,1680991397967 in 195 msec 2023-04-08 22:03:21,170 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-04-08 22:03:21,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, ASSIGN in 359 msec 2023-04-08 22:03:21,172 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:21,173 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991401172"}]},"ts":"1680991401172"} 2023-04-08 22:03:21,175 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-04-08 22:03:21,178 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:21,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 449 msec 2023-04-08 22:03:21,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:21,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,269 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,269 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,270 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:21,270 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:21,270 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,271 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,271 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,272 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,272 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,272 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,273 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,306 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,307 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,312 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,312 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,312 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,313 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,313 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,313 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,313 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,314 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,315 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:21,315 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,315 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:21,315 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:21,317 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:03:21,317 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,317 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,317 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,317 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,318 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,318 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,387 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/9cb0eb89042343fcbe595aa836192178 2023-04-08 22:03:21,456 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/75839e1b2f3a4edfbd6743df00849ceb 2023-04-08 22:03:21,468 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/9cb0eb89042343fcbe595aa836192178 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178 2023-04-08 22:03:21,479 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:21,482 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/75839e1b2f3a4edfbd6743df00849ceb as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb 2023-04-08 22:03:21,493 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:21,496 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 179ms, sequenceid=6, compaction requested=false 2023-04-08 22:03:21,497 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,499 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:21,499 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,500 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:21,500 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:21,500 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,500 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,500 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,502 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,502 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,503 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,503 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:21,505 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:21,505 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a4bd13[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:21,505 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,508 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,509 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:21,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,513 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,513 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,513 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,514 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:21,521 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:21,521 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:21,521 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:21,521 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:21,521 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,521 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,526 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,527 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,527 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,529 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,530 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,530 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,543 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,543 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,542 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,543 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,543 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:21,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,544 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,545 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:21,546 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:21,646 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:21,651 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:21,682 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:21,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:21,685 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,686 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,686 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,686 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:21,686 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:21,687 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,687 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,688 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:21,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,690 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,690 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,691 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,691 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,691 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,691 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,691 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,692 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,692 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:21,692 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,692 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,692 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,693 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,693 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:21,693 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,693 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,698 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,711 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/ee6960c6085b41ea8cda76296ca50f17 2023-04-08 22:03:21,743 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/0fc9be4358b44d628196fded3370a942 2023-04-08 22:03:21,753 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/ee6960c6085b41ea8cda76296ca50f17 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17 2023-04-08 22:03:21,761 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:21,763 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/0fc9be4358b44d628196fded3370a942 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942 2023-04-08 22:03:21,772 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:21,773 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 81ms, sequenceid=19, compaction requested=false 2023-04-08 22:03:21,774 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:21,774 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,774 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:21,775 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:21,775 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,775 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,775 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,776 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,776 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,777 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,777 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:21,779 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21ca28f1[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:21,779 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:21,779 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,780 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,780 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:21,780 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,780 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,780 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:21,780 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,780 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,781 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:21,781 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:21,781 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:21,781 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:21,782 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,782 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,783 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,783 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,783 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:21,784 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,783 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,784 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,784 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,784 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:21,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,787 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:21,789 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:21,789 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:21,789 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:21,789 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:21,789 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,789 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,798 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:21,798 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,799 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,800 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:21,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,800 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:21,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,802 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,803 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,803 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:21,811 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,812 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:21,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:21,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:21,813 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:21,913 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:21,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:21,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:21,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:21,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,948 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:21,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:21,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:21,949 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,949 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,950 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,950 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,950 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:21,950 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,951 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,950 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,951 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:21,951 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,951 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:21,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,952 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,952 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:21,953 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,952 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,953 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:21,954 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,954 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:21,954 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:21,954 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:21,955 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:21,955 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:21,955 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:21,955 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:21,955 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:21,964 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,964 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,965 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,965 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:21,965 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,965 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:21,975 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/5e25bc7bfddc4940931512c0d58e295e 2023-04-08 22:03:21,999 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/57202da0706f4902b5ce3e5612b0b7c7 2023-04-08 22:03:22,009 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/5e25bc7bfddc4940931512c0d58e295e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e 2023-04-08 22:03:22,017 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:22,019 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/57202da0706f4902b5ce3e5612b0b7c7 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7 2023-04-08 22:03:22,027 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:22,029 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 74ms, sequenceid=32, compaction requested=true 2023-04-08 22:03:22,029 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:22,029 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,029 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:22,030 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:22,030 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,030 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,030 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,031 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,031 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,032 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,032 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:22,034 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:22,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7aca119b[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:22,034 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,035 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,035 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:22,037 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,037 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,037 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,038 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,038 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,038 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,038 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,038 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,038 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,039 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,042 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:22,042 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:22,042 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:22,042 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:22,042 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:22,043 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,043 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,048 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,052 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,052 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,059 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,059 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,060 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,059 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,060 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:22,060 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,059 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,060 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:22,060 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,161 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:22,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:22,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:22,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:22,190 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,190 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,190 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,191 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:22,191 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:22,192 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,192 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,193 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,193 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,193 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,194 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:22,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,194 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,195 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,195 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,195 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,195 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,195 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,198 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,199 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,199 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,200 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,206 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:22,206 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,206 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,206 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:22,206 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:22,207 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:22,208 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,208 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,209 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,209 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,209 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,228 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/f0d5e254624e4a9a808e5e4c03e605f8 2023-04-08 22:03:22,259 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 2023-04-08 22:03:22,268 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/f0d5e254624e4a9a808e5e4c03e605f8 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8 2023-04-08 22:03:22,276 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:22,278 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 2023-04-08 22:03:22,287 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:22,289 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 81ms, sequenceid=45, compaction requested=true 2023-04-08 22:03:22,289 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:22,289 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,289 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:22,291 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:22,291 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,291 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,291 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,293 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,294 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:22,297 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:22,297 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@408bcced[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:22,297 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:22,298 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,298 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:22,299 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,299 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,299 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,300 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,300 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,300 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,300 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,301 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,301 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,301 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:22,304 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:22,304 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:22,304 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:22,304 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:22,304 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,305 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,306 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,306 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,306 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,307 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,307 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,309 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,309 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,309 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,312 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,312 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,312 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,313 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:22,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,312 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,313 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,313 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,313 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,314 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,414 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:22,416 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:22,443 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:22,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:22,446 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,446 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,446 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,447 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:22,447 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:22,447 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,447 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,448 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,448 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,448 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,448 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,449 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,449 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,449 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,450 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,450 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,450 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,452 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,452 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,452 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,455 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,453 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,455 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,455 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:22,455 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:22,455 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,455 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:22,455 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:22,464 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,469 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,471 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,471 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,474 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/e983e954676840bfa6ccde5aab92d6e8 2023-04-08 22:03:22,503 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/3acbd0752e06418db2b8be54aea5a4c7 2023-04-08 22:03:22,512 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/e983e954676840bfa6ccde5aab92d6e8 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8 2023-04-08 22:03:22,520 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:22,522 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/3acbd0752e06418db2b8be54aea5a4c7 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7 2023-04-08 22:03:22,530 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:22,532 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 76ms, sequenceid=58, compaction requested=true 2023-04-08 22:03:22,532 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:22,532 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,532 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:22,533 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:22,533 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,533 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,533 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,534 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,534 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,535 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,535 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:22,536 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6364bdca[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:22,536 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:22,537 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,538 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:22,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,538 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:22,539 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,539 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,539 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,540 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,540 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:22,546 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:22,546 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:22,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:22,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:22,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,546 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:22,550 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,550 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,550 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,550 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,550 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,551 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,551 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,552 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,552 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,553 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,553 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,554 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,560 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,560 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:22,561 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,560 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,561 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,561 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,561 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,661 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:22,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:22,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:22,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:22,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,693 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,693 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-04-08 22:03:22,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-04-08 22:03:22,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,695 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,696 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,696 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:22,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,696 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,697 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,697 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,697 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,697 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,698 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,697 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:22,699 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-04-08 22:03:22,700 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. started... 2023-04-08 22:03:22,699 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:22,700 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,700 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,700 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:22,701 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,715 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/b28c818794b3464e82e096cb642da181 2023-04-08 22:03:22,742 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/12259374ffc5432ab3729b96e08b6b70 2023-04-08 22:03:22,750 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/b28c818794b3464e82e096cb642da181 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181 2023-04-08 22:03:22,758 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:22,760 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/12259374ffc5432ab3729b96e08b6b70 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70 2023-04-08 22:03:22,768 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:22,769 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 69ms, sequenceid=71, compaction requested=true 2023-04-08 22:03:22,769 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:22,769 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:22,769 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:22,770 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:22,770 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,770 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-04-08 22:03:22,771 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-04-08 22:03:22,772 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,772 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,772 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,772 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:22,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-04-08 22:03:22,774 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13199cf7[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:22,774 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:22,775 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,775 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,775 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,775 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,775 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,776 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,776 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:22,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:22,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-04-08 22:03:22,776 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:22,777 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,777 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,777 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,778 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,778 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-04-08 22:03:22,778 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,778 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,778 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,778 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,779 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-04-08 22:03:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,781 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,782 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,783 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:22,783 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:22,783 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-04-08 22:03:22,784 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:22,784 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:22,784 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,784 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:22,789 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,789 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,789 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:22,789 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:22,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,789 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,789 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:22,790 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,790 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,791 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:22,791 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:22,792 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:22,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,801 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,801 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-04-08 22:03:22,801 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:22,801 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:22,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:22,801 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:22,802 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,801 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,802 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,802 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:22,803 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:22,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,803 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-04-08 22:03:22,903 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:22,905 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:22,931 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,940 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:22,941 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:22,941 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:22,941 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:22,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, UNASSIGN}] 2023-04-08 22:03:22,951 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, UNASSIGN 2023-04-08 22:03:22,952 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:22,952 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991402952"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991402952"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991402952"}]},"ts":"1680991402952"} 2023-04-08 22:03:22,955 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:23,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, disabling compactions & flushes 2023-04-08 22:03:23,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:23,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:23,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. after waiting 0 ms 2023-04-08 22:03:23,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:23,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:03:23,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/89427bb255a643b6a0e8c80b0eea35cd 2023-04-08 22:03:23,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/714e8462d7014fe6925d9face01e9061 2023-04-08 22:03:23,172 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf1/89427bb255a643b6a0e8c80b0eea35cd as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd 2023-04-08 22:03:23,179 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:23,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.tmp/cf2/714e8462d7014fe6925d9face01e9061 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061 2023-04-08 22:03:23,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:23,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 in 76ms, sequenceid=82, compaction requested=true 2023-04-08 22:03:23,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:03:23,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:23,213 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. 2023-04-08 22:03:23,213 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 9ad6c05dacbf0ddb8d22dd1a8d21f5f2: 2023-04-08 22:03:23,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, regionState=CLOSED 2023-04-08 22:03:23,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991403216"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991403216"}]},"ts":"1680991403216"} 2023-04-08 22:03:23,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-04-08 22:03:23,222 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, server=jenkins-hbase20.apache.org,41749,1680991397967 in 264 msec 2023-04-08 22:03:23,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-04-08 22:03:23,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, UNASSIGN in 273 msec 2023-04-08 22:03:23,244 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, threads=14 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,245 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,246 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,246 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,246 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,246 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,250 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,250 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,250 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,250 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,291 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70, top=true 2023-04-08 22:03:23,316 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8, top=true 2023-04-08 22:03:23,317 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7, top=true 2023-04-08 22:03:23,320 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181, top=true 2023-04-08 22:03:23,320 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd, top=true 2023-04-08 22:03:23,321 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061, top=true 2023-04-08 22:03:23,329 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70 for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,329 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,346 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7 for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,354 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,359 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8 for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,361 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,372 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181 for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,377 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,374 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,379 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,372 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/.splits/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061 for child: 906521a2f5315a45a80af68b91b05565, parent: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,383 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,458 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,484 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,502 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,502 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,510 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,511 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,813 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,878 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17 for region: 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:23,879 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 9ad6c05dacbf0ddb8d22dd1a8d21f5f2 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:23,929 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:23,931 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:23,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1680991403934"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1680991403934"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1680991403934"}]},"ts":"1680991403934"} 2023-04-08 22:03:23,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991403934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991403934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991403934"}]},"ts":"1680991403934"} 2023-04-08 22:03:23,934 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991403934"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991403934"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991403934"}]},"ts":"1680991403934"} 2023-04-08 22:03:23,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, ASSIGN}] 2023-04-08 22:03:23,980 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, ASSIGN 2023-04-08 22:03:23,981 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:23,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, ASSIGN 2023-04-08 22:03:23,983 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:24,131 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:03:24,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=96ecd1c4fcf2c0e9e4913ef6a785f1b7, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,134 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404133"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991404133"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991404133"}]},"ts":"1680991404133"} 2023-04-08 22:03:24,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=15, state=RUNNABLE; OpenRegionProcedure 96ecd1c4fcf2c0e9e4913ef6a785f1b7, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:24,141 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=906521a2f5315a45a80af68b91b05565, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,141 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404141"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991404141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991404141"}]},"ts":"1680991404141"} 2023-04-08 22:03:24,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=16, state=RUNNABLE; OpenRegionProcedure 906521a2f5315a45a80af68b91b05565, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:24,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 906521a2f5315a45a80af68b91b05565, NAME => 'testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.', STARTKEY => '26', ENDKEY => ''} 2023-04-08 22:03:24,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:24,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:24,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,296 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,297 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1 2023-04-08 22:03:24,297 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1 2023-04-08 22:03:24,298 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 906521a2f5315a45a80af68b91b05565 columnFamilyName cf1 2023-04-08 22:03:24,332 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-top 2023-04-08 22:03:24,340 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-top 2023-04-08 22:03:24,348 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-top 2023-04-08 22:03:24,356 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-top 2023-04-08 22:03:24,365 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd 2023-04-08 22:03:24,371 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181 2023-04-08 22:03:24,378 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8 2023-04-08 22:03:24,379 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(310): Store=906521a2f5315a45a80af68b91b05565/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:24,379 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,380 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2 2023-04-08 22:03:24,380 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2 2023-04-08 22:03:24,381 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 906521a2f5315a45a80af68b91b05565 columnFamilyName cf2 2023-04-08 22:03:24,399 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-top 2023-04-08 22:03:24,406 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-top 2023-04-08 22:03:24,413 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-top 2023-04-08 22:03:24,421 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-top 2023-04-08 22:03:24,428 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70 2023-04-08 22:03:24,437 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7 2023-04-08 22:03:24,444 DEBUG [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061 2023-04-08 22:03:24,444 INFO [StoreOpener-906521a2f5315a45a80af68b91b05565-1] regionserver.HStore(310): Store=906521a2f5315a45a80af68b91b05565/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:24,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:24,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:24,452 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 906521a2f5315a45a80af68b91b05565; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63107012, jitterRate=-0.05963224172592163}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:24,452 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 906521a2f5315a45a80af68b91b05565: 2023-04-08 22:03:24,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565., pid=18, masterSystemTime=1680991404289 2023-04-08 22:03:24,455 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,456 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:24,456 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:24,456 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:24,459 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,459 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,459 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1912): 906521a2f5315a45a80af68b91b05565/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:24,459 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1912): 906521a2f5315a45a80af68b91b05565/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:24,459 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 906521a2f5315a45a80af68b91b05565/cf2 in testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,459 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 906521a2f5315a45a80af68b91b05565/cf1 in testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,459 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:24,460 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/.tmp, totalSize=34.9 K 2023-04-08 22:03:24,460 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/.tmp, totalSize=34.9 K 2023-04-08 22:03:24,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,461 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 96ecd1c4fcf2c0e9e4913ef6a785f1b7, NAME => 'testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.', STARTKEY => '', ENDKEY => '26'} 2023-04-08 22:03:24,461 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=906521a2f5315a45a80af68b91b05565, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404461"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991404461"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991404461"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991404461"}]},"ts":"1680991404461"} 2023-04-08 22:03:24,463 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991401238 2023-04-08 22:03:24,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:24,463 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991401238 2023-04-08 22:03:24,464 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:24,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,465 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,465 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991401653 2023-04-08 22:03:24,465 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991401653 2023-04-08 22:03:24,466 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991401916 2023-04-08 22:03:24,467 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991401916 2023-04-08 22:03:24,467 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,467 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991402163 2023-04-08 22:03:24,468 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991402417 2023-04-08 22:03:24,469 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1 2023-04-08 22:03:24,469 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991402163 2023-04-08 22:03:24,470 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991402417 2023-04-08 22:03:24,469 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991402664 2023-04-08 22:03:24,469 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1 2023-04-08 22:03:24,470 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991402664 2023-04-08 22:03:24,470 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991402906 2023-04-08 22:03:24,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=16 2023-04-08 22:03:24,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=16, state=SUCCESS; OpenRegionProcedure 906521a2f5315a45a80af68b91b05565, server=jenkins-hbase20.apache.org,41749,1680991397967 in 323 msec 2023-04-08 22:03:24,471 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991402906 2023-04-08 22:03:24,471 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 96ecd1c4fcf2c0e9e4913ef6a785f1b7 columnFamilyName cf1 2023-04-08 22:03:24,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, ASSIGN in 493 msec 2023-04-08 22:03:24,489 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-bottom 2023-04-08 22:03:24,507 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-bottom 2023-04-08 22:03:24,512 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] throttle.PressureAwareThroughputController(145): 906521a2f5315a45a80af68b91b05565#cf1#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:24,514 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] throttle.PressureAwareThroughputController(145): 906521a2f5315a45a80af68b91b05565#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:24,520 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-bottom 2023-04-08 22:03:24,530 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-bottom 2023-04-08 22:03:24,534 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(310): Store=96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:24,534 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,536 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2 2023-04-08 22:03:24,536 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2 2023-04-08 22:03:24,537 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 96ecd1c4fcf2c0e9e4913ef6a785f1b7 columnFamilyName cf2 2023-04-08 22:03:24,552 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/.tmp/cf1/4256376291d242aeb58d5eb967c3c516 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/4256376291d242aeb58d5eb967c3c516 2023-04-08 22:03:24,569 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-bottom 2023-04-08 22:03:24,571 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/.tmp/cf2/73c50c76a84e4b3c9935b0265e5e62ed as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/73c50c76a84e4b3c9935b0265e5e62ed 2023-04-08 22:03:24,578 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 906521a2f5315a45a80af68b91b05565/cf1 of 906521a2f5315a45a80af68b91b05565 into 4256376291d242aeb58d5eb967c3c516(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:24,578 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 906521a2f5315a45a80af68b91b05565: 2023-04-08 22:03:24,578 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565., storeName=906521a2f5315a45a80af68b91b05565/cf1, priority=9, startTime=1680991404455; duration=0sec 2023-04-08 22:03:24,578 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-bottom 2023-04-08 22:03:24,579 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,583 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 906521a2f5315a45a80af68b91b05565/cf2 of 906521a2f5315a45a80af68b91b05565 into 73c50c76a84e4b3c9935b0265e5e62ed(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:24,583 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 906521a2f5315a45a80af68b91b05565: 2023-04-08 22:03:24,583 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565., storeName=906521a2f5315a45a80af68b91b05565/cf2, priority=9, startTime=1680991404456; duration=0sec 2023-04-08 22:03:24,583 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,584 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-bottom 2023-04-08 22:03:24,591 DEBUG [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-bottom 2023-04-08 22:03:24,591 INFO [StoreOpener-96ecd1c4fcf2c0e9e4913ef6a785f1b7-1] regionserver.HStore(310): Store=96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:24,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,596 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:24,598 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:24,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 96ecd1c4fcf2c0e9e4913ef6a785f1b7; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67486210, jitterRate=0.005622893571853638}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:24,599 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 96ecd1c4fcf2c0e9e4913ef6a785f1b7: 2023-04-08 22:03:24,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7., pid=17, masterSystemTime=1680991404289 2023-04-08 22:03:24,600 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,601 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-08 22:03:24,601 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-04-08 22:03:24,601 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:24,601 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,601 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,602 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1912): 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:24,602 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1912): 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:24,602 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2 in testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,602 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1 in testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,602 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/.tmp, totalSize=19.9 K 2023-04-08 22:03:24,603 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/.tmp, totalSize=19.9 K 2023-04-08 22:03:24,604 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991401238 2023-04-08 22:03:24,604 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:24,605 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991401238 2023-04-08 22:03:24,605 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991401653 2023-04-08 22:03:24,605 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991401653 2023-04-08 22:03:24,606 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=96ecd1c4fcf2c0e9e4913ef6a785f1b7, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,606 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991401916 2023-04-08 22:03:24,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404606"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991404606"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991404606"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991404606"}]},"ts":"1680991404606"} 2023-04-08 22:03:24,606 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991401916 2023-04-08 22:03:24,607 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991402163 2023-04-08 22:03:24,607 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991402163 2023-04-08 22:03:24,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=15 2023-04-08 22:03:24,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=15, state=SUCCESS; OpenRegionProcedure 96ecd1c4fcf2c0e9e4913ef6a785f1b7, server=jenkins-hbase20.apache.org,41749,1680991397967 in 473 msec 2023-04-08 22:03:24,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-04-08 22:03:24,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, ASSIGN in 635 msec 2023-04-08 22:03:24,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 in 1.6860 sec 2023-04-08 22:03:24,625 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] throttle.PressureAwareThroughputController(145): 96ecd1c4fcf2c0e9e4913ef6a785f1b7#cf2#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:24,628 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] throttle.PressureAwareThroughputController(145): 96ecd1c4fcf2c0e9e4913ef6a785f1b7#cf1#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:24,645 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/.tmp/cf2/a77958c42d4d461b970c4727007fc3ac as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/a77958c42d4d461b970c4727007fc3ac 2023-04-08 22:03:24,648 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/.tmp/cf1/b02ee4386cf6480a86ab6169d7b5da0f as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/b02ee4386cf6480a86ab6169d7b5da0f 2023-04-08 22:03:24,653 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2 of 96ecd1c4fcf2c0e9e4913ef6a785f1b7 into a77958c42d4d461b970c4727007fc3ac(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:24,653 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 96ecd1c4fcf2c0e9e4913ef6a785f1b7: 2023-04-08 22:03:24,653 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7., storeName=96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2, priority=12, startTime=1680991404601; duration=0sec 2023-04-08 22:03:24,653 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,659 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1 of 96ecd1c4fcf2c0e9e4913ef6a785f1b7 into b02ee4386cf6480a86ab6169d7b5da0f(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:24,659 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 96ecd1c4fcf2c0e9e4913ef6a785f1b7: 2023-04-08 22:03:24,660 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7., storeName=96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1, priority=12, startTime=1680991404600; duration=0sec 2023-04-08 22:03:24,660 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:24,709 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41749] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:38532 deadline: 1680991464699, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2. is not online on jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,833 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-04-08 22:03:24,846 INFO [Listener at localhost.localdomain/46567] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-08 22:03:24,849 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-04-08 22:03:24,878 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:24,887 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-04-08 22:03:24,891 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionUnevenDaughter 2023-04-08 22:03:24,898 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:24,903 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991404903"}]},"ts":"1680991404903"} 2023-04-08 22:03:24,905 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-04-08 22:03:24,909 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-08 22:03:24,934 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-04-08 22:03:24,936 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, UNASSIGN}] 2023-04-08 22:03:24,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, UNASSIGN 2023-04-08 22:03:24,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=906521a2f5315a45a80af68b91b05565, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404939"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991404939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991404939"}]},"ts":"1680991404939"} 2023-04-08 22:03:24,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 906521a2f5315a45a80af68b91b05565, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:24,945 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, UNASSIGN 2023-04-08 22:03:24,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=96ecd1c4fcf2c0e9e4913ef6a785f1b7, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:24,947 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991404946"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991404946"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991404946"}]},"ts":"1680991404946"} 2023-04-08 22:03:24,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 96ecd1c4fcf2c0e9e4913ef6a785f1b7, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:25,013 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-08 22:03:25,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:25,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 96ecd1c4fcf2c0e9e4913ef6a785f1b7, disabling compactions & flushes 2023-04-08 22:03:25,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:25,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:25,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. after waiting 0 ms 2023-04-08 22:03:25,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:25,102 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-bottom] to archive 2023-04-08 22:03:25,104 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:25,110 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,113 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,116 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,118 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,145 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-bottom] to archive 2023-04-08 22:03:25,147 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:25,149 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,153 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,155 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,157 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,164 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:03:25,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:25,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7. 2023-04-08 22:03:25,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 96ecd1c4fcf2c0e9e4913ef6a785f1b7: 2023-04-08 22:03:25,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:25,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:25,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 906521a2f5315a45a80af68b91b05565, disabling compactions & flushes 2023-04-08 22:03:25,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:25,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:25,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. after waiting 0 ms 2023-04-08 22:03:25,168 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:25,169 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=96ecd1c4fcf2c0e9e4913ef6a785f1b7, regionState=CLOSED 2023-04-08 22:03:25,169 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991405169"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991405169"}]},"ts":"1680991405169"} 2023-04-08 22:03:25,170 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd] to archive 2023-04-08 22:03:25,172 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:25,176 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-04-08 22:03:25,177 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 96ecd1c4fcf2c0e9e4913ef6a785f1b7, server=jenkins-hbase20.apache.org,41749,1680991397967 in 224 msec 2023-04-08 22:03:25,178 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/9cb0eb89042343fcbe595aa836192178.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,179 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=96ecd1c4fcf2c0e9e4913ef6a785f1b7, UNASSIGN in 241 msec 2023-04-08 22:03:25,180 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/ee6960c6085b41ea8cda76296ca50f17.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,182 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/5e25bc7bfddc4940931512c0d58e295e.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,185 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/f0d5e254624e4a9a808e5e4c03e605f8.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,187 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-e983e954676840bfa6ccde5aab92d6e8 2023-04-08 22:03:25,189 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-b28c818794b3464e82e096cb642da181 2023-04-08 22:03:25,192 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-89427bb255a643b6a0e8c80b0eea35cd 2023-04-08 22:03:25,194 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061] to archive 2023-04-08 22:03:25,197 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:25,201 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/75839e1b2f3a4edfbd6743df00849ceb.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,203 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/0fc9be4358b44d628196fded3370a942.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,206 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/57202da0706f4902b5ce3e5612b0b7c7.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,208 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3.9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,211 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-3acbd0752e06418db2b8be54aea5a4c7 2023-04-08 22:03:25,213 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-12259374ffc5432ab3729b96e08b6b70 2023-04-08 22:03:25,215 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-08 22:03:25,216 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/testSplitTableRegionUnevenDaughter=9ad6c05dacbf0ddb8d22dd1a8d21f5f2-714e8462d7014fe6925d9face01e9061 2023-04-08 22:03:25,217 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:03:25,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:03:25,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:25,239 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565. 2023-04-08 22:03:25,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 906521a2f5315a45a80af68b91b05565: 2023-04-08 22:03:25,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:25,243 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=906521a2f5315a45a80af68b91b05565, regionState=CLOSED 2023-04-08 22:03:25,244 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1680991405243"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991405243"}]},"ts":"1680991405243"} 2023-04-08 22:03:25,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-04-08 22:03:25,251 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 906521a2f5315a45a80af68b91b05565, server=jenkins-hbase20.apache.org,41749,1680991397967 in 305 msec 2023-04-08 22:03:25,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-04-08 22:03:25,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=906521a2f5315a45a80af68b91b05565, UNASSIGN in 315 msec 2023-04-08 22:03:25,259 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991405257"}]},"ts":"1680991405257"} 2023-04-08 22:03:25,261 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-04-08 22:03:25,263 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-04-08 22:03:25,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 372 msec 2023-04-08 22:03:25,312 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-04-08 22:03:25,313 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-04-08 22:03:25,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-04-08 22:03:25,518 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-04-08 22:03:25,524 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,534 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,536 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,537 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-08 22:03:25,545 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,545 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:25,545 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:25,548 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/recovered.edits] 2023-04-08 22:03:25,548 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/recovered.edits] 2023-04-08 22:03:25,548 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/recovered.edits] 2023-04-08 22:03:25,552 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/b02ee4386cf6480a86ab6169d7b5da0f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf1/b02ee4386cf6480a86ab6169d7b5da0f 2023-04-08 22:03:25,552 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/5e25bc7bfddc4940931512c0d58e295e 2023-04-08 22:03:25,552 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/4256376291d242aeb58d5eb967c3c516 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf1/4256376291d242aeb58d5eb967c3c516 2023-04-08 22:03:25,553 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/89427bb255a643b6a0e8c80b0eea35cd 2023-04-08 22:03:25,554 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/a77958c42d4d461b970c4727007fc3ac to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/cf2/a77958c42d4d461b970c4727007fc3ac 2023-04-08 22:03:25,555 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/9cb0eb89042343fcbe595aa836192178 2023-04-08 22:03:25,555 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/73c50c76a84e4b3c9935b0265e5e62ed to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/cf2/73c50c76a84e4b3c9935b0265e5e62ed 2023-04-08 22:03:25,557 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/b28c818794b3464e82e096cb642da181 2023-04-08 22:03:25,559 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7/recovered.edits/90.seqid 2023-04-08 22:03:25,560 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/e983e954676840bfa6ccde5aab92d6e8 2023-04-08 22:03:25,560 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565/recovered.edits/90.seqid 2023-04-08 22:03:25,560 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/96ecd1c4fcf2c0e9e4913ef6a785f1b7 2023-04-08 22:03:25,561 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:25,562 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/ee6960c6085b41ea8cda76296ca50f17 2023-04-08 22:03:25,563 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf1/f0d5e254624e4a9a808e5e4c03e605f8 2023-04-08 22:03:25,566 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/0fc9be4358b44d628196fded3370a942 2023-04-08 22:03:25,567 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/12259374ffc5432ab3729b96e08b6b70 2023-04-08 22:03:25,568 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/3acbd0752e06418db2b8be54aea5a4c7 2023-04-08 22:03:25,570 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/57202da0706f4902b5ce3e5612b0b7c7 2023-04-08 22:03:25,572 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/6500e9cd1df44dbf8e98ae972c5a0ff3 2023-04-08 22:03:25,574 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/714e8462d7014fe6925d9face01e9061 2023-04-08 22:03:25,575 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/cf2/75839e1b2f3a4edfbd6743df00849ceb 2023-04-08 22:03:25,578 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2/recovered.edits/85.seqid 2023-04-08 22:03:25,579 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionUnevenDaughter/9ad6c05dacbf0ddb8d22dd1a8d21f5f2 2023-04-08 22:03:25,579 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-04-08 22:03:25,582 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,602 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-04-08 22:03:25,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-08 22:03:25,647 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-04-08 22:03:25,649 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,649 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-04-08 22:03:25,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991405649"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:25,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991405649"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:25,649 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991405649"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:25,653 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:03:25,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9ad6c05dacbf0ddb8d22dd1a8d21f5f2, NAME => 'testSplitTableRegionUnevenDaughter,,1680991400729.9ad6c05dacbf0ddb8d22dd1a8d21f5f2.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 96ecd1c4fcf2c0e9e4913ef6a785f1b7, NAME => 'testSplitTableRegionUnevenDaughter,,1680991402932.96ecd1c4fcf2c0e9e4913ef6a785f1b7.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 906521a2f5315a45a80af68b91b05565, NAME => 'testSplitTableRegionUnevenDaughter,26,1680991402932.906521a2f5315a45a80af68b91b05565.', STARTKEY => '26', ENDKEY => ''}] 2023-04-08 22:03:25,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-04-08 22:03:25,653 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991405653"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:25,656 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-04-08 22:03:25,658 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:25,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 133 msec 2023-04-08 22:03:25,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-04-08 22:03:25,841 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-04-08 22:03:25,855 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=391 (was 360) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:58006 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57972 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42128 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57998 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42196 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50424 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57976 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase20:41749-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57784 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42110 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57808 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57730 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:54810 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42144 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57724 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50378 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57992 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=826 (was 691) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=238 (was 224) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 165), AvailableMemoryMB=4313 (was 4414) 2023-04-08 22:03:25,868 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=391, OpenFileDescriptor=826, MaxFileDescriptor=60000, SystemLoadAverage=238, ProcessCount=165, AvailableMemoryMB=4312 2023-04-08 22:03:25,868 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:25,872 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:25,873 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:25,875 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:25,877 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:25,878 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:25,880 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:25,880 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e empty. 2023-04-08 22:03:25,881 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:25,881 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-04-08 22:03:25,892 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:25,893 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6f9219e0fb8fc810d2587ec30edba29e, NAME => 'testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:25,905 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:25,906 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 6f9219e0fb8fc810d2587ec30edba29e, disabling compactions & flushes 2023-04-08 22:03:25,906 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:25,906 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:25,906 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. after waiting 0 ms 2023-04-08 22:03:25,906 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:25,906 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:25,906 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:25,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:25,910 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991405909"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991405909"}]},"ts":"1680991405909"} 2023-04-08 22:03:25,912 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:25,913 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:25,913 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991405913"}]},"ts":"1680991405913"} 2023-04-08 22:03:25,915 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-04-08 22:03:25,918 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:25,919 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:25,919 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:25,919 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:25,919 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:25,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, ASSIGN}] 2023-04-08 22:03:25,921 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, ASSIGN 2023-04-08 22:03:25,922 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:26,072 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:26,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991406074"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991406074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991406074"}]},"ts":"1680991406074"} 2023-04-08 22:03:26,076 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:26,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6f9219e0fb8fc810d2587ec30edba29e, NAME => 'testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:26,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:26,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,234 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,236 DEBUG [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1 2023-04-08 22:03:26,236 DEBUG [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1 2023-04-08 22:03:26,236 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f9219e0fb8fc810d2587ec30edba29e columnFamilyName cf1 2023-04-08 22:03:26,237 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] regionserver.HStore(310): Store=6f9219e0fb8fc810d2587ec30edba29e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:26,237 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,239 DEBUG [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2 2023-04-08 22:03:26,239 DEBUG [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2 2023-04-08 22:03:26,240 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6f9219e0fb8fc810d2587ec30edba29e columnFamilyName cf2 2023-04-08 22:03:26,241 INFO [StoreOpener-6f9219e0fb8fc810d2587ec30edba29e-1] regionserver.HStore(310): Store=6f9219e0fb8fc810d2587ec30edba29e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:26,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:26,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:26,247 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:26,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 6f9219e0fb8fc810d2587ec30edba29e; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61611412, jitterRate=-0.08191841840744019}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:26,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:26,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e., pid=27, masterSystemTime=1680991406228 2023-04-08 22:03:26,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,252 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991406252"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991406252"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991406252"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991406252"}]},"ts":"1680991406252"} 2023-04-08 22:03:26,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-04-08 22:03:26,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 in 178 msec 2023-04-08 22:03:26,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-04-08 22:03:26,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, ASSIGN in 339 msec 2023-04-08 22:03:26,262 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:26,263 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991406262"}]},"ts":"1680991406262"} 2023-04-08 22:03:26,264 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-04-08 22:03:26,267 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:26,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 394 msec 2023-04-08 22:03:26,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:26,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:26,390 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:26,390 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,391 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:26,391 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:26,392 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,392 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,393 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,393 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:26,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,394 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,394 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,395 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,395 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,396 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,396 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,396 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,396 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,396 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,402 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,402 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,402 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,402 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,403 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,403 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:26,402 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:26,403 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:26,403 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,404 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:03:26,404 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,406 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,408 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,408 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,408 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,409 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,409 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,417 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/29927edc9a49435a88202bb85cfaf57c 2023-04-08 22:03:26,438 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/3bb694b62fed463bb5bf0fc2c102f326 2023-04-08 22:03:26,444 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/29927edc9a49435a88202bb85cfaf57c as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c 2023-04-08 22:03:26,450 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:26,451 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/3bb694b62fed463bb5bf0fc2c102f326 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 2023-04-08 22:03:26,457 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:26,458 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 54ms, sequenceid=6, compaction requested=false 2023-04-08 22:03:26,459 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-04-08 22:03:26,459 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:26,459 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,459 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:26,459 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:26,459 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,459 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,459 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,461 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,461 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,461 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,461 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:26,463 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@397501c6[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:26,463 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:26,463 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,464 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,464 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:26,465 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,465 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,465 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,465 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,467 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,467 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,467 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,467 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,468 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,468 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:26,471 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:26,471 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:26,471 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:26,471 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:26,471 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:26,471 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,481 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,482 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,482 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,484 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,484 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,484 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,489 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,490 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:26,490 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:26,490 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,490 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,489 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,490 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:26,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,490 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:26,491 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,492 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,492 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,492 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,592 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:26,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:26,615 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:26,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:26,617 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:26,617 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:26,617 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,618 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:26,618 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:26,618 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,618 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,619 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,619 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,619 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,619 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,619 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,620 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:26,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,619 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,621 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,621 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,621 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,622 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,622 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,622 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,621 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,623 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,622 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,623 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,623 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,623 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,623 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,624 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:26,624 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,624 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,624 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:26,624 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,624 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,624 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:26,624 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:26,625 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,625 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,625 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,625 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,626 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,626 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,636 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/d63a55e32a4848f68d17eab1de84ec14 2023-04-08 22:03:26,651 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/1aab016aa4a545ceb73bcfe335326edb 2023-04-08 22:03:26,657 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/d63a55e32a4848f68d17eab1de84ec14 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 2023-04-08 22:03:26,663 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:26,664 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/1aab016aa4a545ceb73bcfe335326edb as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb 2023-04-08 22:03:26,669 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:26,670 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 46ms, sequenceid=19, compaction requested=false 2023-04-08 22:03:26,671 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:26,671 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,671 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:26,671 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:26,671 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,671 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,671 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,672 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,672 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,673 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,673 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:26,675 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27075ea1[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:26,675 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:26,675 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,676 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,676 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:26,676 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:26,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:26,676 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:26,678 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,678 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,678 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,678 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,679 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,679 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,679 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,679 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,679 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:26,683 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:26,683 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:26,683 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:26,683 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:26,683 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:26,683 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,692 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,692 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,693 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,693 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,693 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,697 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,697 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,700 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:26,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:26,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,702 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:26,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,802 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:26,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:26,824 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:26,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:26,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:26,826 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:26,826 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,827 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:26,827 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:26,828 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,828 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,829 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,829 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,829 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,830 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,830 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,830 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:26,831 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,831 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:26,831 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,831 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:26,831 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,831 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:26,832 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:26,833 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,833 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:26,833 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:26,833 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:26,833 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,833 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,833 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:26,834 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,834 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,834 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,834 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,835 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,835 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,849 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/f4070994346f435ab3500ecc81acf990 2023-04-08 22:03:26,869 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 2023-04-08 22:03:26,874 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/f4070994346f435ab3500ecc81acf990 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 2023-04-08 22:03:26,880 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:26,881 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 2023-04-08 22:03:26,886 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:26,887 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 54ms, sequenceid=32, compaction requested=true 2023-04-08 22:03:26,887 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:26,888 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:26,888 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:26,888 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:26,888 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,888 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:26,888 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:26,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,889 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,889 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,889 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:26,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:26,891 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61cb3d66[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:26,891 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:26,891 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,892 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,892 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,892 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:26,893 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:26,893 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:26,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,895 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,895 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,895 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,895 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:26,895 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,896 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,896 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:26,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:26,900 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:26,900 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:26,900 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:26,900 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:26,900 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:26,901 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:26,908 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,909 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,909 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,910 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,910 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,910 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:26,912 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:26,913 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,913 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:26,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:26,917 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:26,918 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:26,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:26,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:26,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,918 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:26,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:26,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:26,920 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:26,920 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:27,020 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:27,022 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:27,045 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:27,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:27,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:27,048 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:27,048 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:27,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:27,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,049 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,053 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,053 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,053 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,053 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,053 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,055 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,055 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,055 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,056 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,058 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,058 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:27,058 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:27,058 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,058 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:27,059 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:27,059 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-08 22:03:27,059 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-04-08 22:03:27,060 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:27,060 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,065 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,069 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/3f4f748919ae4896a7e21e93b1136453 2023-04-08 22:03:27,084 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/dea972bc06f44269b839f3261b1f40cc 2023-04-08 22:03:27,091 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/3f4f748919ae4896a7e21e93b1136453 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 2023-04-08 22:03:27,096 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:27,097 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/dea972bc06f44269b839f3261b1f40cc as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc 2023-04-08 22:03:27,103 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:27,104 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 46ms, sequenceid=45, compaction requested=true 2023-04-08 22:03:27,105 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:27,105 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,105 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:27,105 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:27,105 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,105 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,105 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,106 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,106 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,106 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,106 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:27,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e6aeb8c[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:27,108 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:27,108 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:27,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,109 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,112 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,112 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,112 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,112 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:27,117 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:27,117 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:27,117 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:27,117 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:27,117 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:27,117 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,126 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,126 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,126 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,126 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,128 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,128 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,128 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,133 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:27,134 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,134 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,134 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:27,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:27,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,135 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,235 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:27,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:27,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:27,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:27,262 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:27,262 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:27,262 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,262 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:27,262 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:27,263 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,263 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,264 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,264 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,264 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,264 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,265 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,266 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,266 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,266 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,266 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,266 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,266 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,266 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,266 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,267 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,267 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,267 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,270 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,271 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,271 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:27,271 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:27,271 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,271 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:27,271 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:27,275 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,276 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,276 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,286 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/9fd10ad9ab77443baffaf250622f63b5 2023-04-08 22:03:27,302 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/be5e3154fad543ff818a72bf78452f63 2023-04-08 22:03:27,308 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/9fd10ad9ab77443baffaf250622f63b5 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 2023-04-08 22:03:27,314 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:27,315 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/be5e3154fad543ff818a72bf78452f63 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 2023-04-08 22:03:27,321 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:27,323 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 52ms, sequenceid=58, compaction requested=true 2023-04-08 22:03:27,323 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:27,323 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,323 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:27,323 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:27,323 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,323 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,323 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,324 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,324 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,324 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,325 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,325 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:27,327 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f00a6b6[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:27,327 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:27,327 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,328 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,328 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,329 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,330 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,329 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,330 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:27,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:27,333 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:27,333 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:27,333 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:27,333 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:27,333 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,335 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,335 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,336 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,336 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,336 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,337 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,338 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,338 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:27,348 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,348 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,348 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,349 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,350 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,350 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,350 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,350 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,449 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:27,451 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:27,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:27,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:27,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-04-08 22:03:27,474 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-04-08 22:03:27,474 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,474 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-04-08 22:03:27,474 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-04-08 22:03:27,475 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,475 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,476 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,476 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,476 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,477 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-04-08 22:03:27,477 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,477 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,477 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,477 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,478 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-04-08 22:03:27,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,478 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:27,478 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,478 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. started... 2023-04-08 22:03:27,478 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:27,478 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,489 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/3046b4f26acf420bbe19d52d01010d3e 2023-04-08 22:03:27,505 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/dc32a3361ad14c319065014c711b8447 2023-04-08 22:03:27,510 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/3046b4f26acf420bbe19d52d01010d3e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e 2023-04-08 22:03:27,515 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:27,516 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/dc32a3361ad14c319065014c711b8447 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 2023-04-08 22:03:27,522 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:27,524 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 45ms, sequenceid=71, compaction requested=true 2023-04-08 22:03:27,524 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:27,524 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:27,524 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:27,524 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:27,524 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,524 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-04-08 22:03:27,524 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-04-08 22:03:27,525 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,525 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,525 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,525 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-04-08 22:03:27,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32ef07a1[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:27,527 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-04-08 22:03:27,527 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,528 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:27,528 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:27,529 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-04-08 22:03:27,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,529 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,530 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,530 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-04-08 22:03:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:03:27,532 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-04-08 22:03:27,532 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-04-08 22:03:27,532 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:27,532 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:27,532 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-04-08 22:03:27,532 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:27,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,533 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,534 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:27,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,534 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,535 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,536 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:27,536 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-04-08 22:03:27,536 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,538 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-04-08 22:03:27,539 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:27,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:27,538 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,539 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,538 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:27,538 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-04-08 22:03:27,539 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:27,539 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:27,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:27,539 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:27,539 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,540 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,540 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:27,540 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-04-08 22:03:27,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:27,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:27,540 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-04-08 22:03:27,639 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:27,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:27,912 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-08 22:03:27,914 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:27,915 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:27,916 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:27,916 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:27,916 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:27,917 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:27,917 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:27,917 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:27,918 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:28,165 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,167 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:28,167 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:28,167 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:28,169 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-04-08 22:03:28,170 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:28,170 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:28,170 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:28,171 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:28,171 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:28,175 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-08 22:03:28,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:28,178 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:28,178 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:28,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:28,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:28,181 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,181 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,182 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:28,183 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,183 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:28,183 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,183 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,183 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,184 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-04-08 22:03:28,185 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:28,186 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:28,187 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991438187 2023-04-08 22:03:28,188 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,188 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:28,193 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,194 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,194 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:28,194 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991468194 2023-04-08 22:03:28,194 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991528194 2023-04-08 22:03:28,194 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-08 22:03:28,194 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:28,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:28,197 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:28,197 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:28,444 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,444 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:28,445 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:28,445 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:28,446 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:28,447 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-08 22:03:28,448 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:28,448 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:28,448 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:28,449 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:28,449 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,449 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:28,449 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,450 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,450 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:28,450 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,450 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:28,450 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,451 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,451 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,451 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:28,451 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:28,452 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:28,452 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991438452 2023-04-08 22:03:28,454 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,454 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:28,457 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,458 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,458 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:28,458 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991468458 2023-04-08 22:03:28,458 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991528458 2023-04-08 22:03:28,458 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-08 22:03:28,458 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:28,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN}] 2023-04-08 22:03:28,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:28,461 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:28,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:28,708 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,708 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:28,709 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:28,709 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:28,710 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:28,710 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:28,710 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:28,710 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:28,710 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:28,711 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:28,712 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-08 22:03:28,713 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:28,713 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:28,713 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:28,713 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:28,714 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,714 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:28,714 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,715 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,715 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:28,715 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,715 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:28,715 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,716 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,716 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,716 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:28,717 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:28,717 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:28,718 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991438717 2023-04-08 22:03:28,719 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,719 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:28,723 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,723 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,723 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:28,723 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991468723 2023-04-08 22:03:28,723 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991528723 2023-04-08 22:03:28,723 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:28,724 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN}] 2023-04-08 22:03:28,724 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:28,727 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:28,728 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:28,728 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991408728"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991408728"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991408728"}]},"ts":"1680991408728"} 2023-04-08 22:03:28,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:28,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:28,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:28,730 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:28,730 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:28,974 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:28,974 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:28,975 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:28,975 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:28,977 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:28,978 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:28,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:28,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:28,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:28,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:28,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:28,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:28,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:28,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:28,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:28,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:28,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:28,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:28,983 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-08 22:03:28,984 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:28,984 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:28,984 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e to restore RIT 2023-04-08 22:03:28,985 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991438985 2023-04-08 22:03:28,986 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:28,986 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:28,988 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,988 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:28,988 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:28,988 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991468988 2023-04-08 22:03:28,989 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991528988 2023-04-08 22:03:28,989 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:28,989 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:28,989 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991408989"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991408989"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991408989"}]},"ts":"1680991408989"} 2023-04-08 22:03:28,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:28,991 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:29,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 6f9219e0fb8fc810d2587ec30edba29e, disabling compactions & flushes 2023-04-08 22:03:29,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:29,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:29,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. after waiting 0 ms 2023-04-08 22:03:29,145 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:29,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 6f9219e0fb8fc810d2587ec30edba29e 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:03:29,156 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 2023-04-08 22:03:29,172 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b 2023-04-08 22:03:29,177 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 2023-04-08 22:03:29,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:29,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/.tmp/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b 2023-04-08 22:03:29,188 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:29,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6f9219e0fb8fc810d2587ec30edba29e in 44ms, sequenceid=82, compaction requested=true 2023-04-08 22:03:29,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:03:29,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:29,212 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. 2023-04-08 22:03:29,212 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 6f9219e0fb8fc810d2587ec30edba29e: 2023-04-08 22:03:29,215 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,215 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED 2023-04-08 22:03:29,215 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991409215"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991409215"}]},"ts":"1680991409215"} 2023-04-08 22:03:29,217 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:29,217 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:29,217 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:29,239 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:29,239 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:29,240 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:29,240 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:29,240 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:29,242 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:29,242 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-08 22:03:29,242 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-04-08 22:03:29,242 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:29,242 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:29,243 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:29,243 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:29,243 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,243 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:29,243 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,244 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,244 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:29,245 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,245 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,245 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:29,245 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,245 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:29,245 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:29,246 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:29,247 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:29,247 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e to restore RIT 2023-04-08 22:03:29,247 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991439247 2023-04-08 22:03:29,248 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,248 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:29,251 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,251 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,251 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:29,252 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991469252 2023-04-08 22:03:29,252 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991529252 2023-04-08 22:03:29,252 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:29,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,252 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991409252"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991409252"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991409252"}]},"ts":"1680991409252"} 2023-04-08 22:03:29,254 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:29,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-04-08 22:03:29,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 in 263 msec 2023-04-08 22:03:29,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:29,258 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:29,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:29,502 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,502 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:29,502 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:29,503 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:29,503 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:29,503 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:29,503 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:29,503 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:29,503 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:29,504 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:29,505 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:29,506 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-08 22:03:29,506 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:29,506 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:29,507 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:29,507 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:29,507 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:29,507 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,508 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:29,508 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,508 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,508 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:29,509 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,509 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,509 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:29,509 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:29,510 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:29,510 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:29,511 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:29,511 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e to restore RIT 2023-04-08 22:03:29,511 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991439511 2023-04-08 22:03:29,512 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,512 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:29,514 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,514 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,515 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:29,515 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991469515 2023-04-08 22:03:29,515 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991529515 2023-04-08 22:03:29,515 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:29,515 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,517 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:29,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-04-08 22:03:29,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN in 793 msec 2023-04-08 22:03:29,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:29,523 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:29,524 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:29,765 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,765 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:29,765 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:29,765 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:29,766 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:29,766 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:29,766 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:29,766 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:29,766 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:29,767 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:29,768 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:29,768 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:29,768 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:29,768 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:29,769 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:29,769 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:29,769 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:29,770 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,770 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:29,770 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,770 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:29,770 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:29,771 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:29,771 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:29,772 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:29,773 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:29,773 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991439773 2023-04-08 22:03:29,774 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,774 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:29,776 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,777 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:29,777 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:29,777 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991469777 2023-04-08 22:03:29,777 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991529777 2023-04-08 22:03:29,777 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:29,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,777 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:29,782 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:29,793 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6f9219e0fb8fc810d2587ec30edba29e, threads=14 2023-04-08 22:03:29,794 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,794 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,795 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,795 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,796 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,796 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,797 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,797 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,797 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,797 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,797 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,798 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,804 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,804 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,975 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:29,982 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,002 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,007 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,007 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,007 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,012 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,013 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,026 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,030 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,030 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,038 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,041 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,041 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,041 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6f9219e0fb8fc810d2587ec30edba29e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:30,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:30,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:30,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:30,277 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:30,278 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:30,278 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:30,279 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:30,279 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:30,280 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-08 22:03:30,280 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:30,281 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:30,281 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:30,281 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:30,281 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:30,282 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:30,282 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,282 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:30,283 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,283 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,283 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:30,284 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,284 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:30,284 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:30,285 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:30,285 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,285 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,285 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,285 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:30,285 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:30,285 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:30,286 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:30,287 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:30,289 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991440289 2023-04-08 22:03:30,290 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,290 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:30,293 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:30,293 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:30,293 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:30,293 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991470293 2023-04-08 22:03:30,294 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991530294 2023-04-08 22:03:30,294 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:30,303 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6f9219e0fb8fc810d2587ec30edba29e, threads=14 2023-04-08 22:03:30,305 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,305 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,310 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,310 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,310 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,311 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,311 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,311 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,311 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,311 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,312 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,314 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,314 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,315 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,489 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,530 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,534 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,538 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,538 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,538 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,542 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,548 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,564 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,570 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,574 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,585 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,589 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,589 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb for region: 6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:30,590 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6f9219e0fb8fc810d2587ec30edba29e Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:30,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:30,635 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:30,638 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:30,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:30,638 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:30,638 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:30,669 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:03:30,794 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,794 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:30,794 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:30,794 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:30,794 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:30,795 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:30,795 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:30,795 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:30,795 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:30,796 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:30,797 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-08 22:03:30,797 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:30,797 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:30,797 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:30,798 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:30,798 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:30,798 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:30,798 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,799 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:30,799 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,799 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:30,799 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:30,800 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,800 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,800 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:30,800 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:30,801 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:30,801 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:30,802 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:30,802 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991440802 2023-04-08 22:03:30,803 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:30,804 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,806 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:30,807 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:30,807 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:30,807 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991470807 2023-04-08 22:03:30,807 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991530807 2023-04-08 22:03:30,807 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:30,810 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:30,813 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:30,813 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:30,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:30,813 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:31,057 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:31,057 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:31,057 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:31,058 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:31,058 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:31,058 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:31,058 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:31,058 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:31,058 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:31,059 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:31,060 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-08 22:03:31,060 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:31,060 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:31,060 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:31,060 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:31,061 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:31,062 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:31,062 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,062 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN 2023-04-08 22:03:31,062 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,063 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:31,063 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,063 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,063 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,063 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6f9219e0fb8fc810d2587ec30edba29e, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:31,063 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6f9219e0fb8fc810d2587ec30edba29e, server=jenkins-hbase20.apache.org,41749,1680991397967 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:31,063 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:31,064 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:31,064 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:31,065 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991441065 2023-04-08 22:03:31,066 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:31,066 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,068 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:31,068 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=6f9219e0fb8fc810d2587ec30edba29e, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:31,068 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:31,069 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991471069 2023-04-08 22:03:31,069 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991531069 2023-04-08 22:03:31,069 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:31,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991411071"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1680991411071"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1680991411071"}]},"ts":"1680991411071"} 2023-04-08 22:03:31,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991411071"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991411071"}]},"ts":"1680991411071"} 2023-04-08 22:03:31,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411071"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991411071"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991411071"}]},"ts":"1680991411071"} 2023-04-08 22:03:31,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, ASSIGN}] 2023-04-08 22:03:31,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, ASSIGN 2023-04-08 22:03:31,082 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:31,087 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, ASSIGN 2023-04-08 22:03:31,109 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:31,232 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:03:31,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=68c1c7892be17242d3b4efe192ab38a1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411233"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991411233"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991411233"}]},"ts":"1680991411233"} 2023-04-08 22:03:31,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 68c1c7892be17242d3b4efe192ab38a1, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:31,239 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f059ab2aed67beb6b09f8d73a44d767b, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,239 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411239"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991411239"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991411239"}]},"ts":"1680991411239"} 2023-04-08 22:03:31,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure f059ab2aed67beb6b09f8d73a44d767b, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:31,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f059ab2aed67beb6b09f8d73a44d767b, NAME => 'testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.', STARTKEY => '', ENDKEY => '41'} 2023-04-08 22:03:31,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:31,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:31,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,394 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,395 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1 2023-04-08 22:03:31,395 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1 2023-04-08 22:03:31,395 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f059ab2aed67beb6b09f8d73a44d767b columnFamilyName cf1 2023-04-08 22:03:31,415 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-bottom 2023-04-08 22:03:31,421 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-bottom 2023-04-08 22:03:31,427 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-bottom 2023-04-08 22:03:31,433 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-bottom 2023-04-08 22:03:31,439 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-bottom 2023-04-08 22:03:31,445 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-bottom 2023-04-08 22:03:31,450 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-bottom 2023-04-08 22:03:31,450 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(310): Store=f059ab2aed67beb6b09f8d73a44d767b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:31,450 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,451 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2 2023-04-08 22:03:31,451 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2 2023-04-08 22:03:31,451 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f059ab2aed67beb6b09f8d73a44d767b columnFamilyName cf2 2023-04-08 22:03:31,468 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-bottom 2023-04-08 22:03:31,474 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-bottom 2023-04-08 22:03:31,480 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-bottom 2023-04-08 22:03:31,485 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-bottom 2023-04-08 22:03:31,490 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-bottom 2023-04-08 22:03:31,497 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-bottom 2023-04-08 22:03:31,502 DEBUG [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-bottom 2023-04-08 22:03:31,502 INFO [StoreOpener-f059ab2aed67beb6b09f8d73a44d767b-1] regionserver.HStore(310): Store=f059ab2aed67beb6b09f8d73a44d767b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:31,503 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,504 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,506 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:31,507 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:31,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened f059ab2aed67beb6b09f8d73a44d767b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69756038, jitterRate=0.039445966482162476}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:31,508 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for f059ab2aed67beb6b09f8d73a44d767b: 2023-04-08 22:03:31,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b., pid=34, masterSystemTime=1680991411388 2023-04-08 22:03:31,509 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:31,510 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:31,510 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:31,510 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:31,510 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,510 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,510 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1912): f059ab2aed67beb6b09f8d73a44d767b/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:31,510 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1912): f059ab2aed67beb6b09f8d73a44d767b/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:31,510 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f059ab2aed67beb6b09f8d73a44d767b/cf1 in testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,510 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f059ab2aed67beb6b09f8d73a44d767b/cf2 in testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,511 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/.tmp, totalSize=34.9 K 2023-04-08 22:03:31,511 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/.tmp, totalSize=34.9 K 2023-04-08 22:03:31,511 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991406379 2023-04-08 22:03:31,511 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991406379 2023-04-08 22:03:31,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:31,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,512 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991406594 2023-04-08 22:03:31,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 68c1c7892be17242d3b4efe192ab38a1, NAME => 'testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.', STARTKEY => '41', ENDKEY => ''} 2023-04-08 22:03:31,512 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991406594 2023-04-08 22:03:31,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:31,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,512 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991406804 2023-04-08 22:03:31,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:31,513 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991406804 2023-04-08 22:03:31,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,513 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991407023 2023-04-08 22:03:31,513 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991407023 2023-04-08 22:03:31,513 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=f059ab2aed67beb6b09f8d73a44d767b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,513 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991407238 2023-04-08 22:03:31,514 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411513"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991411513"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991411513"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991411513"}]},"ts":"1680991411513"} 2023-04-08 22:03:31,514 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991407238 2023-04-08 22:03:31,514 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991407451 2023-04-08 22:03:31,514 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,514 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991407642 2023-04-08 22:03:31,514 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991407451 2023-04-08 22:03:31,515 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991407642 2023-04-08 22:03:31,516 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1 2023-04-08 22:03:31,516 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1 2023-04-08 22:03:31,517 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68c1c7892be17242d3b4efe192ab38a1 columnFamilyName cf1 2023-04-08 22:03:31,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-04-08 22:03:31,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure f059ab2aed67beb6b09f8d73a44d767b, server=jenkins-hbase20.apache.org,41749,1680991397967 in 275 msec 2023-04-08 22:03:31,523 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, ASSIGN in 442 msec 2023-04-08 22:03:31,546 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] throttle.PressureAwareThroughputController(145): f059ab2aed67beb6b09f8d73a44d767b#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:31,546 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] throttle.PressureAwareThroughputController(145): f059ab2aed67beb6b09f8d73a44d767b#cf1#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:31,552 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-top 2023-04-08 22:03:31,565 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-top 2023-04-08 22:03:31,571 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-top 2023-04-08 22:03:31,575 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/.tmp/cf1/ee6165d3c4794f84b30bf02178ffba96 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/ee6165d3c4794f84b30bf02178ffba96 2023-04-08 22:03:31,577 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-top 2023-04-08 22:03:31,581 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f059ab2aed67beb6b09f8d73a44d767b/cf1 of f059ab2aed67beb6b09f8d73a44d767b into ee6165d3c4794f84b30bf02178ffba96(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:31,581 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f059ab2aed67beb6b09f8d73a44d767b: 2023-04-08 22:03:31,581 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b., storeName=f059ab2aed67beb6b09f8d73a44d767b/cf1, priority=9, startTime=1680991411509; duration=0sec 2023-04-08 22:03:31,581 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:31,582 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/.tmp/cf2/5dbd9798fea44f9aa27def391413e53b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/5dbd9798fea44f9aa27def391413e53b 2023-04-08 22:03:31,583 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-top 2023-04-08 22:03:31,587 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-top 2023-04-08 22:03:31,588 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f059ab2aed67beb6b09f8d73a44d767b/cf2 of f059ab2aed67beb6b09f8d73a44d767b into 5dbd9798fea44f9aa27def391413e53b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:31,588 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f059ab2aed67beb6b09f8d73a44d767b: 2023-04-08 22:03:31,588 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b., storeName=f059ab2aed67beb6b09f8d73a44d767b/cf2, priority=9, startTime=1680991411510; duration=0sec 2023-04-08 22:03:31,588 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:31,593 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-top 2023-04-08 22:03:31,593 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(310): Store=68c1c7892be17242d3b4efe192ab38a1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:31,593 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,594 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2 2023-04-08 22:03:31,595 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2 2023-04-08 22:03:31,595 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 68c1c7892be17242d3b4efe192ab38a1 columnFamilyName cf2 2023-04-08 22:03:31,613 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-top 2023-04-08 22:03:31,618 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-top 2023-04-08 22:03:31,622 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-top 2023-04-08 22:03:31,627 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-top 2023-04-08 22:03:31,631 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-top 2023-04-08 22:03:31,636 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-top 2023-04-08 22:03:31,641 DEBUG [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-top 2023-04-08 22:03:31,641 INFO [StoreOpener-68c1c7892be17242d3b4efe192ab38a1-1] regionserver.HStore(310): Store=68c1c7892be17242d3b4efe192ab38a1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:31,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:31,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:31,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 68c1c7892be17242d3b4efe192ab38a1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59841484, jitterRate=-0.10829240083694458}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:31,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 68c1c7892be17242d3b4efe192ab38a1: 2023-04-08 22:03:31,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1., pid=33, masterSystemTime=1680991411388 2023-04-08 22:03:31,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:31,649 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:31,649 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:31,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:31,649 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,649 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,649 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1912): 68c1c7892be17242d3b4efe192ab38a1/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:31,649 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1912): 68c1c7892be17242d3b4efe192ab38a1/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:31,649 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 68c1c7892be17242d3b4efe192ab38a1/cf2 in testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,649 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 68c1c7892be17242d3b4efe192ab38a1/cf1 in testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,649 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/.tmp, totalSize=34.9 K 2023-04-08 22:03:31,650 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/.tmp, totalSize=34.9 K 2023-04-08 22:03:31,650 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991406379 2023-04-08 22:03:31,650 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991406379 2023-04-08 22:03:31,651 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting 1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991406594 2023-04-08 22:03:31,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,651 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991406594 2023-04-08 22:03:31,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:31,651 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991406804 2023-04-08 22:03:31,651 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991406804 2023-04-08 22:03:31,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=68c1c7892be17242d3b4efe192ab38a1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991407023 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991407023 2023-04-08 22:03:31,652 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411651"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991411651"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991411651"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991411651"}]},"ts":"1680991411651"} 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991407238 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991407238 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991407451 2023-04-08 22:03:31,652 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991407451 2023-04-08 22:03:31,653 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991407642 2023-04-08 22:03:31,653 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting 833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991407642 2023-04-08 22:03:31,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-04-08 22:03:31,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 68c1c7892be17242d3b4efe192ab38a1, server=jenkins-hbase20.apache.org,41749,1680991397967 in 418 msec 2023-04-08 22:03:31,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=32, resume processing ppid=28 2023-04-08 22:03:31,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, ASSIGN in 580 msec 2023-04-08 22:03:31,662 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 in 3.7450 sec 2023-04-08 22:03:31,667 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] throttle.PressureAwareThroughputController(145): 68c1c7892be17242d3b4efe192ab38a1#cf1#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:31,667 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] throttle.PressureAwareThroughputController(145): 68c1c7892be17242d3b4efe192ab38a1#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:31,688 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/.tmp/cf2/130770d1158d4ae290fb420efc98c04e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/130770d1158d4ae290fb420efc98c04e 2023-04-08 22:03:31,700 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/.tmp/cf1/96ee56afd26240cdb281fb53831a84bb as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/96ee56afd26240cdb281fb53831a84bb 2023-04-08 22:03:31,702 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 68c1c7892be17242d3b4efe192ab38a1/cf2 of 68c1c7892be17242d3b4efe192ab38a1 into 130770d1158d4ae290fb420efc98c04e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:31,702 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 68c1c7892be17242d3b4efe192ab38a1: 2023-04-08 22:03:31,702 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1., storeName=68c1c7892be17242d3b4efe192ab38a1/cf2, priority=9, startTime=1680991411649; duration=0sec 2023-04-08 22:03:31,702 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:31,707 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 68c1c7892be17242d3b4efe192ab38a1/cf1 of 68c1c7892be17242d3b4efe192ab38a1 into 96ee56afd26240cdb281fb53831a84bb(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:31,707 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 68c1c7892be17242d3b4efe192ab38a1: 2023-04-08 22:03:31,707 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1., storeName=68c1c7892be17242d3b4efe192ab38a1/cf1, priority=9, startTime=1680991411648; duration=0sec 2023-04-08 22:03:31,708 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:31,820 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41749] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:38532 deadline: 1680991471820, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e. is not online on jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,933 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:03:31,944 INFO [Listener at localhost.localdomain/46567] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-08 22:03:31,946 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:03:31,966 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:31,968 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-04-08 22:03:31,968 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitWithoutPONR 2023-04-08 22:03:31,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:31,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-08 22:03:31,973 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991411973"}]},"ts":"1680991411973"} 2023-04-08 22:03:31,975 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-04-08 22:03:31,976 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-04-08 22:03:31,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, UNASSIGN}] 2023-04-08 22:03:31,979 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, UNASSIGN 2023-04-08 22:03:31,979 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=68c1c7892be17242d3b4efe192ab38a1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,980 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411979"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991411979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991411979"}]},"ts":"1680991411979"} 2023-04-08 22:03:31,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 68c1c7892be17242d3b4efe192ab38a1, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:31,984 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, UNASSIGN 2023-04-08 22:03:31,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f059ab2aed67beb6b09f8d73a44d767b, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:31,985 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991411985"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991411985"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991411985"}]},"ts":"1680991411985"} 2023-04-08 22:03:31,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure f059ab2aed67beb6b09f8d73a44d767b, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:32,074 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-08 22:03:32,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:32,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 68c1c7892be17242d3b4efe192ab38a1, disabling compactions & flushes 2023-04-08 22:03:32,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:32,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:32,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. after waiting 0 ms 2023-04-08 22:03:32,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:32,146 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-top] to archive 2023-04-08 22:03:32,147 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:32,149 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,150 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,151 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,153 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,154 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,155 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,157 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,159 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-top] to archive 2023-04-08 22:03:32,160 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:32,162 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,163 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,164 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,166 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,167 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,168 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,169 DEBUG [StoreCloser-testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,173 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:03:32,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:32,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1. 2023-04-08 22:03:32,174 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 68c1c7892be17242d3b4efe192ab38a1: 2023-04-08 22:03:32,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:32,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:32,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing f059ab2aed67beb6b09f8d73a44d767b, disabling compactions & flushes 2023-04-08 22:03:32,176 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:32,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:32,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. after waiting 0 ms 2023-04-08 22:03:32,176 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:32,176 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=68c1c7892be17242d3b4efe192ab38a1, regionState=CLOSED 2023-04-08 22:03:32,176 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991412176"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991412176"}]},"ts":"1680991412176"} 2023-04-08 22:03:32,177 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01-bottom] to archive 2023-04-08 22:03:32,178 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:32,180 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-04-08 22:03:32,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 68c1c7892be17242d3b4efe192ab38a1, server=jenkins-hbase20.apache.org,41749,1680991397967 in 197 msec 2023-04-08 22:03:32,181 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,182 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=68c1c7892be17242d3b4efe192ab38a1, UNASSIGN in 204 msec 2023-04-08 22:03:32,183 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,184 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,186 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,187 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,189 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,191 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b-bottom] to archive 2023-04-08 22:03:32,191 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:32,193 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,194 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,195 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,196 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,197 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,198 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,199 DEBUG [StoreCloser-testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:03:32,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:32,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b. 2023-04-08 22:03:32,204 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for f059ab2aed67beb6b09f8d73a44d767b: 2023-04-08 22:03:32,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:32,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=f059ab2aed67beb6b09f8d73a44d767b, regionState=CLOSED 2023-04-08 22:03:32,207 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991412206"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991412206"}]},"ts":"1680991412206"} 2023-04-08 22:03:32,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-04-08 22:03:32,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure f059ab2aed67beb6b09f8d73a44d767b, server=jenkins-hbase20.apache.org,41749,1680991397967 in 221 msec 2023-04-08 22:03:32,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-04-08 22:03:32,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=f059ab2aed67beb6b09f8d73a44d767b, UNASSIGN in 233 msec 2023-04-08 22:03:32,213 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991412213"}]},"ts":"1680991412213"} 2023-04-08 22:03:32,214 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-04-08 22:03:32,215 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-04-08 22:03:32,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 247 msec 2023-04-08 22:03:32,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-04-08 22:03:32,276 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-04-08 22:03:32,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitWithoutPONR 2023-04-08 22:03:32,278 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,280 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,281 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-04-08 22:03:32,289 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,289 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:32,289 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:32,291 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/recovered.edits] 2023-04-08 22:03:32,291 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/recovered.edits] 2023-04-08 22:03:32,291 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/recovered.edits] 2023-04-08 22:03:32,294 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/ee6165d3c4794f84b30bf02178ffba96 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf1/ee6165d3c4794f84b30bf02178ffba96 2023-04-08 22:03:32,294 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/.regioninfo to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/.regioninfo 2023-04-08 22:03:32,295 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/29927edc9a49435a88202bb85cfaf57c 2023-04-08 22:03:32,296 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3046b4f26acf420bbe19d52d01010d3e 2023-04-08 22:03:32,297 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/5dbd9798fea44f9aa27def391413e53b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/cf2/5dbd9798fea44f9aa27def391413e53b 2023-04-08 22:03:32,298 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/3f4f748919ae4896a7e21e93b1136453 2023-04-08 22:03:32,298 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,299 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/833ffbb2c8be40daa6d26afc6e2b8b01 2023-04-08 22:03:32,299 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,300 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/.regioninfo to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/.regioninfo 2023-04-08 22:03:32,300 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/9fd10ad9ab77443baffaf250622f63b5 2023-04-08 22:03:32,301 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,302 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/d63a55e32a4848f68d17eab1de84ec14 2023-04-08 22:03:32,302 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,303 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf1/f4070994346f435ab3500ecc81acf990 2023-04-08 22:03:32,303 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/29927edc9a49435a88202bb85cfaf57c.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,303 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,304 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,304 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/3046b4f26acf420bbe19d52d01010d3e.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,306 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,306 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/3f4f748919ae4896a7e21e93b1136453.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,306 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/1aab016aa4a545ceb73bcfe335326edb 2023-04-08 22:03:32,307 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/833ffbb2c8be40daa6d26afc6e2b8b01.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,308 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/3bb694b62fed463bb5bf0fc2c102f326 2023-04-08 22:03:32,308 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/9fd10ad9ab77443baffaf250622f63b5.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,309 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,310 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/be5e3154fad543ff818a72bf78452f63 2023-04-08 22:03:32,310 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/d63a55e32a4848f68d17eab1de84ec14.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,310 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,311 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b 2023-04-08 22:03:32,311 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf1/f4070994346f435ab3500ecc81acf990.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,312 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,313 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dc32a3361ad14c319065014c711b8447 2023-04-08 22:03:32,313 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,314 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/dea972bc06f44269b839f3261b1f40cc 2023-04-08 22:03:32,315 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/1aab016aa4a545ceb73bcfe335326edb.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,315 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,316 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/cf2/ef06c5dc59a94e53a60ce5ec76f58c68 2023-04-08 22:03:32,317 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/3bb694b62fed463bb5bf0fc2c102f326.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,317 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,319 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/be5e3154fad543ff818a72bf78452f63.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,319 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/68c1c7892be17242d3b4efe192ab38a1/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,326 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e/recovered.edits/85.seqid 2023-04-08 22:03:32,327 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/d1b9b0fdc4de4b1a97925d60cef02f1b.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,327 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,328 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/96ee56afd26240cdb281fb53831a84bb to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf1/96ee56afd26240cdb281fb53831a84bb 2023-04-08 22:03:32,329 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/dc32a3361ad14c319065014c711b8447.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,330 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/dea972bc06f44269b839f3261b1f40cc.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,331 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/130770d1158d4ae290fb420efc98c04e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/cf2/130770d1158d4ae290fb420efc98c04e 2023-04-08 22:03:32,332 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/f059ab2aed67beb6b09f8d73a44d767b/cf2/ef06c5dc59a94e53a60ce5ec76f58c68.6f9219e0fb8fc810d2587ec30edba29e 2023-04-08 22:03:32,335 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1/recovered.edits/90.seqid 2023-04-08 22:03:32,336 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b/recovered.edits/90.seqid 2023-04-08 22:03:32,336 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:32,337 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitWithoutPONR/f059ab2aed67beb6b09f8d73a44d767b 2023-04-08 22:03:32,337 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-04-08 22:03:32,340 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,343 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-04-08 22:03:32,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-04-08 22:03:32,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-04-08 22:03:32,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991412348"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:32,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991412348"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:32,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991412348"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:32,350 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:03:32,351 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6f9219e0fb8fc810d2587ec30edba29e, NAME => 'testSplitWithoutPONR,,1680991405873.6f9219e0fb8fc810d2587ec30edba29e.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f059ab2aed67beb6b09f8d73a44d767b, NAME => 'testSplitWithoutPONR,,1680991407914.f059ab2aed67beb6b09f8d73a44d767b.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 68c1c7892be17242d3b4efe192ab38a1, NAME => 'testSplitWithoutPONR,41,1680991407914.68c1c7892be17242d3b4efe192ab38a1.', STARTKEY => '41', ENDKEY => ''}] 2023-04-08 22:03:32,351 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-04-08 22:03:32,351 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991412351"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:32,353 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-04-08 22:03:32,354 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:32,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 77 msec 2023-04-08 22:03:32,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-04-08 22:03:32,383 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-04-08 22:03:32,399 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=396 (was 391) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:58006 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42464 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42128 [Waiting for operation #27] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50786 [Waiting for operation #21] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57998 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50424 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57784 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42110 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50750 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:54810 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:48054 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:48014 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:42144 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:48006 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3a7752df-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:57992 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1251953617_15 at /127.0.0.1:50810 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 826) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=225 (was 238), ProcessCount=165 (was 165), AvailableMemoryMB=4195 (was 4312) 2023-04-08 22:03:32,414 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=396, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=225, ProcessCount=165, AvailableMemoryMB=4194 2023-04-08 22:03:32,414 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:32,417 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:32,417 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:32,418 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:32,420 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:32,421 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:32,423 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,424 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 empty. 2023-04-08 22:03:32,424 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,424 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-04-08 22:03:32,437 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:32,438 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 53819c752f224f75f5ed122f97922705, NAME => 'testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:32,448 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:32,449 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 53819c752f224f75f5ed122f97922705, disabling compactions & flushes 2023-04-08 22:03:32,449 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,449 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,449 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. after waiting 0 ms 2023-04-08 22:03:32,449 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,449 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,449 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:32,451 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:32,452 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991412452"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991412452"}]},"ts":"1680991412452"} 2023-04-08 22:03:32,453 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:32,454 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:32,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991412455"}]},"ts":"1680991412455"} 2023-04-08 22:03:32,456 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-04-08 22:03:32,460 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:32,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:32,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:32,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:32,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:32,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN}] 2023-04-08 22:03:32,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:32,465 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,43551,1680991397921; forceNewPlan=false, retain=false 2023-04-08 22:03:32,615 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:32,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,616 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991412616"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991412616"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991412616"}]},"ts":"1680991412616"} 2023-04-08 22:03:32,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:32,771 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,771 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-08 22:03:32,775 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:58842, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-08 22:03:32,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 53819c752f224f75f5ed122f97922705, NAME => 'testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:32,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:32,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:32,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,786 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,788 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1 2023-04-08 22:03:32,788 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1 2023-04-08 22:03:32,788 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53819c752f224f75f5ed122f97922705 columnFamilyName cf1 2023-04-08 22:03:32,789 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(310): Store=53819c752f224f75f5ed122f97922705/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:32,789 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,791 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2 2023-04-08 22:03:32,791 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2 2023-04-08 22:03:32,791 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53819c752f224f75f5ed122f97922705 columnFamilyName cf2 2023-04-08 22:03:32,792 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(310): Store=53819c752f224f75f5ed122f97922705/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:32,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:32,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:32,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:32,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 53819c752f224f75f5ed122f97922705; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70385126, jitterRate=0.04882010817527771}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:32,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:32,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705., pid=43, masterSystemTime=1680991412771 2023-04-08 22:03:32,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,806 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,806 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991412806"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991412806"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991412806"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991412806"}]},"ts":"1680991412806"} 2023-04-08 22:03:32,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-04-08 22:03:32,809 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 in 189 msec 2023-04-08 22:03:32,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-04-08 22:03:32,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN in 349 msec 2023-04-08 22:03:32,813 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:32,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991412813"}]},"ts":"1680991412813"} 2023-04-08 22:03:32,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-04-08 22:03:32,817 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:32,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 400 msec 2023-04-08 22:03:32,922 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-08 22:03:32,926 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:58848, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-08 22:03:32,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:32,935 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:32,936 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:32,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:32,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:32,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:32,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:32,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:32,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:32,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:32,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:32,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:32,937 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:32,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:32,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:32,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:32,939 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:32,940 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,940 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,940 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:32,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:32,946 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:32,946 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:32,946 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:32,947 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:32,947 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:03:32,950 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,950 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,950 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:32,960 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/4ddfb947d30944ef8727e64dc8d9ce96 2023-04-08 22:03:32,976 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/e412af38f4274eb3abf74cbcf0525f2e 2023-04-08 22:03:32,980 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/4ddfb947d30944ef8727e64dc8d9ce96 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 2023-04-08 22:03:32,985 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:32,985 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/e412af38f4274eb3abf74cbcf0525f2e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e 2023-04-08 22:03:32,990 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:32,991 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 44ms, sequenceid=6, compaction requested=false 2023-04-08 22:03:32,991 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-04-08 22:03:32,991 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:32,992 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:32,992 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:32,992 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:32,992 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,992 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:32,992 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:32,993 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,993 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:32,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:32,993 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,993 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:32,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:32,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:32,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:32,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:32,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:32,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:32,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:32,995 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:32,995 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@af72c97[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:32,995 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:32,996 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:32,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:32,996 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:32,996 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:32,997 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:32,997 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,002 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,002 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,002 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,003 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,003 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,006 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,006 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,006 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,006 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,006 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,007 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,009 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,010 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,010 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,011 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,012 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,012 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,013 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,113 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:33,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:33,133 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:33,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:33,135 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:33,135 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,135 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,136 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:33,136 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:33,136 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,136 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,137 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,137 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,138 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,139 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,140 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,140 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:33,140 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:33,140 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,140 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,141 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,153 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 2023-04-08 22:03:33,167 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 2023-04-08 22:03:33,173 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 2023-04-08 22:03:33,177 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:33,178 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 2023-04-08 22:03:33,182 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:33,183 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 43ms, sequenceid=19, compaction requested=false 2023-04-08 22:03:33,184 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:33,184 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:33,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:33,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,185 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:33,187 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63a890ce[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:33,187 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:33,187 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,188 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:33,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,188 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,189 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:33,189 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,189 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,189 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:33,190 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,190 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,190 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,190 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,191 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,193 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,193 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,193 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,195 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,195 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,195 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,196 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,197 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,198 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,198 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,200 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,200 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,201 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,301 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:33,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:33,322 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:33,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:33,324 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:33,324 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,324 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,324 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:33,325 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:33,325 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,325 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,326 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,326 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,326 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,326 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,327 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:33,328 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:33,329 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,329 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:33,329 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:33,329 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,329 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,329 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,329 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,330 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,330 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,340 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/86734e03dea04ababbb025727ff6e644 2023-04-08 22:03:33,352 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/c59f155a385d43459cef6647013d9023 2023-04-08 22:03:33,357 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/86734e03dea04ababbb025727ff6e644 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 2023-04-08 22:03:33,361 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:33,362 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/c59f155a385d43459cef6647013d9023 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 2023-04-08 22:03:33,367 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:33,368 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 39ms, sequenceid=32, compaction requested=true 2023-04-08 22:03:33,368 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:33,369 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,369 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:33,369 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:33,369 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,369 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,369 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,370 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,370 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,370 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,370 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:33,372 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@49b7bf1b[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:33,372 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:33,372 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,373 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:33,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,373 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,373 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:33,374 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,374 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,374 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:33,374 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,374 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,374 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,375 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,377 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,377 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,377 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,377 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,377 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,377 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,379 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,379 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,379 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,379 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,388 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,388 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,388 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,393 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,392 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,393 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,392 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,393 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,393 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,393 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,394 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,493 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:33,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:33,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:33,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:33,514 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:33,514 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,514 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,514 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:33,514 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:33,515 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,515 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,516 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,516 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,516 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,516 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:33,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,517 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,518 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,517 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:33,518 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,518 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:33,518 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,518 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,519 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,529 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/f410c0e5508c4af38d0ff2b472aca6cb 2023-04-08 22:03:33,545 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/542247275f0a4f16bcce858c72fafa9b 2023-04-08 22:03:33,549 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/f410c0e5508c4af38d0ff2b472aca6cb as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb 2023-04-08 22:03:33,554 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:33,555 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/542247275f0a4f16bcce858c72fafa9b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b 2023-04-08 22:03:33,560 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:33,561 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 43ms, sequenceid=45, compaction requested=true 2023-04-08 22:03:33,561 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:33,561 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,561 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:33,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:33,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:33,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:33,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@32f6b83[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:33,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,565 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,565 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,565 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,566 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:33,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:33,566 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:33,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,566 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,567 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,567 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,570 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,570 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,570 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,571 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,572 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,572 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,574 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,574 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,574 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,574 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,576 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,576 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,577 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,577 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,677 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:33,678 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:33,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:33,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:33,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:33,698 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:33,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:33,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,700 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,700 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,701 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,701 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,701 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,702 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,702 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:33,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:33,702 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,703 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,720 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/9a96079839354dd2b7e542ba3c195ad7 2023-04-08 22:03:33,735 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/2f3ce244a8624af5b9a99c31da5d98e4 2023-04-08 22:03:33,740 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/9a96079839354dd2b7e542ba3c195ad7 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 2023-04-08 22:03:33,746 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:33,747 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/2f3ce244a8624af5b9a99c31da5d98e4 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 2023-04-08 22:03:33,752 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:33,753 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 51ms, sequenceid=58, compaction requested=true 2023-04-08 22:03:33,753 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:33,753 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,753 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:33,753 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:33,753 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,754 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,754 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,755 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,755 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,755 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,755 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:33,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@300f6ce5[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:33,757 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:33,757 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,758 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,758 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:33,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:33,758 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:33,760 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,760 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,761 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,761 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,761 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,764 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,764 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,764 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,764 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,764 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,766 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,766 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,768 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,768 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,769 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,772 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,773 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution because node does not exist (not an error) 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,773 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,773 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,773 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,774 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,774 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,774 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,774 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,874 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:33,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:33,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:33,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:33,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-04-08 22:03:33,891 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,891 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,891 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-04-08 22:03:33,891 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:03:33,892 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,892 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,892 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,892 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,893 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,893 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:33,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:33,895 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:33,895 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,894 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,895 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. started... 2023-04-08 22:03:33,895 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,895 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:33,895 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,896 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,896 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,896 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,896 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,896 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,903 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/77a4bee3fd71467d8f0e9a8530729ccd 2023-04-08 22:03:33,916 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/c4990b55a053410b80d0e47034870f30 2023-04-08 22:03:33,920 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/77a4bee3fd71467d8f0e9a8530729ccd as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd 2023-04-08 22:03:33,923 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:33,924 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/c4990b55a053410b80d0e47034870f30 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 2023-04-08 22:03:33,929 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:33,930 INFO [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 35ms, sequenceid=71, compaction requested=true 2023-04-08 22:03:33,931 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:33,931 DEBUG [rs(jenkins-hbase20.apache.org,43551,1680991397921)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:33,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:33,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:33,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-04-08 22:03:33,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-04-08 22:03:33,932 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,932 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,933 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,933 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:33,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-04-08 22:03:33,934 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1706b4ca[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:33,934 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:03:33,934 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,935 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,935 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:33,935 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,935 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:33,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,936 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-04-08 22:03:33,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,937 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-04-08 22:03:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase20.apache.org,43551,1680991397921': 2023-04-08 22:03:33,940 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:03:33,940 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-04-08 22:03:33,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:33,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:33,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-04-08 22:03:33,940 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,941 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:33,942 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,942 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:33,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:33,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,944 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:33,944 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-04-08 22:03:33,944 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-04-08 22:03:33,947 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:33,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:33,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,947 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-04-08 22:03:33,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-04-08 22:03:34,048 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:34,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:34,311 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-08 22:03:34,312 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:34,313 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,314 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,314 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,315 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,315 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:34,316 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:34,316 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:34,564 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:34,564 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:34,565 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:34,566 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-08 22:03:34,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:34,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:34,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:34,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:34,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:34,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:34,570 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,570 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,570 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,570 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:34,571 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:34,571 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:34,571 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991444571 2023-04-08 22:03:34,573 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:34,573 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,576 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:34,576 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:34,576 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:34,576 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991474576 2023-04-08 22:03:34,576 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991534576 2023-04-08 22:03:34,577 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:34,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:34,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:34,580 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:34,580 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:34,827 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:34,827 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:34,828 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:34,829 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-08 22:03:34,829 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:34,830 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:34,830 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:34,830 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:34,830 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,830 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,831 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:34,832 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:34,832 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,832 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:34,832 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,832 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,832 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,832 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-04-08 22:03:34,833 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:34,833 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:34,834 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991444834 2023-04-08 22:03:34,835 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,835 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:34,837 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:34,837 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:34,838 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:34,838 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991474838 2023-04-08 22:03:34,838 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991534838 2023-04-08 22:03:34,838 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:34,838 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:34,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN}] 2023-04-08 22:03:34,840 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:34,840 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:34,840 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:34,841 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:35,088 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,088 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:35,088 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:35,088 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:35,088 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:35,089 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:35,089 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:35,089 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:35,089 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:35,089 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:35,091 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-08 22:03:35,091 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:35,092 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:35,092 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:35,092 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:35,092 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,093 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:35,093 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,093 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,094 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,094 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:35,095 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,095 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,095 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:35,096 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,096 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,096 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,096 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,096 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,096 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-08 22:03:35,097 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:35,098 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:35,098 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991445098 2023-04-08 22:03:35,100 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,100 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:35,103 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:35,103 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:35,103 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:35,103 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991475103 2023-04-08 22:03:35,104 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991535104 2023-04-08 22:03:35,104 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:35,104 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN}] 2023-04-08 22:03:35,104 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:35,107 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,108 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,108 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991415108"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991415108"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991415108"}]},"ts":"1680991415108"} 2023-04-08 22:03:35,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:35,110 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:35,110 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:35,110 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:35,354 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,354 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:35,354 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:35,354 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:35,355 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:35,355 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:35,355 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:35,355 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:35,355 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:35,356 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:35,357 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:35,358 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,358 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:35,358 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,358 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,359 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,359 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:35,359 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,359 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,360 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:35,360 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,360 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,360 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,360 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,360 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,361 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,361 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,361 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,361 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:35,362 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:35,362 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:35,362 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:35,363 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991445362 2023-04-08 22:03:35,363 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:35,363 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,366 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:35,366 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSING, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:35,366 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:35,366 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991475366 2023-04-08 22:03:35,366 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991535366 2023-04-08 22:03:35,367 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:35,367 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,367 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991415367"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991415367"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991415367"}]},"ts":"1680991415367"} 2023-04-08 22:03:35,369 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:35,369 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:35,522 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:35,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 53819c752f224f75f5ed122f97922705, disabling compactions & flushes 2023-04-08 22:03:35,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:35,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:35,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. after waiting 0 ms 2023-04-08 22:03:35,523 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:35,523 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 53819c752f224f75f5ed122f97922705 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:03:35,532 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/9b33a90c122d4134be4f3f7ad403b619 2023-04-08 22:03:35,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/edcfef2570c34079a542d15d61c34580 2023-04-08 22:03:35,553 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/9b33a90c122d4134be4f3f7ad403b619 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 2023-04-08 22:03:35,557 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:35,558 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/edcfef2570c34079a542d15d61c34580 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 2023-04-08 22:03:35,564 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:03:35,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 53819c752f224f75f5ed122f97922705 in 42ms, sequenceid=82, compaction requested=true 2023-04-08 22:03:35,569 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:03:35,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:35,570 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:35,570 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:35,572 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:35,573 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSED 2023-04-08 22:03:35,573 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991415573"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991415573"}]},"ts":"1680991415573"} 2023-04-08 22:03:35,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:35,575 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,575 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:35,575 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:35,617 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,617 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:35,617 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:35,617 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:35,617 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:35,617 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:35,618 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:35,618 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:35,618 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:35,618 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:35,620 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:35,620 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-08 22:03:35,620 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-04-08 22:03:35,620 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:35,621 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:35,621 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:35,621 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:35,622 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,622 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:35,622 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,623 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,623 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,624 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:35,624 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,624 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,624 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:35,625 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,625 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,625 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,625 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,625 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,626 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,626 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,626 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,626 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,626 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,626 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-08 22:03:35,627 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:35,627 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:35,627 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:35,634 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991445634 2023-04-08 22:03:35,635 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,635 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:35,638 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:35,639 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:35,639 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:35,639 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991475639 2023-04-08 22:03:35,639 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991535639 2023-04-08 22:03:35,639 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:35,640 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSED, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,640 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991415639"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991415639"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991415639"}]},"ts":"1680991415639"} 2023-04-08 22:03:35,641 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:35,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-04-08 22:03:35,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 in 272 msec 2023-04-08 22:03:35,645 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:35,645 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,645 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:35,645 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:35,890 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:35,890 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:35,891 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:35,891 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:35,892 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:35,893 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:35,893 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-04-08 22:03:35,893 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:35,894 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:35,894 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:35,894 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:35,895 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:35,895 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,895 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:35,896 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,896 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:35,896 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,897 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:35,897 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,897 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:35,898 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:35,898 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:35,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:35,899 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,899 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,899 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:35,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:35,899 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-08 22:03:35,900 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:35,900 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:35,900 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:35,901 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991445901 2023-04-08 22:03:35,902 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,902 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:35,904 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:35,904 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:35,904 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:35,904 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991475904 2023-04-08 22:03:35,904 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991535904 2023-04-08 22:03:35,904 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:35,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:35,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:35,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-04-08 22:03:35,909 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN in 802 msec 2023-04-08 22:03:35,912 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:35,912 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:35,912 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:35,913 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:36,154 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,155 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:36,155 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:36,155 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:36,157 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:36,157 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:36,157 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:36,157 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:36,157 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:36,158 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:36,159 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:36,159 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:36,159 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:36,160 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:36,160 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:36,160 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:36,160 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:36,161 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,161 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:36,161 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,161 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,161 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,162 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:36,162 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,162 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,162 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:36,163 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:36,163 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:36,164 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:36,164 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:36,166 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991446166 2023-04-08 22:03:36,168 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,168 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:36,170 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:36,171 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:36,171 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:36,171 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991476171 2023-04-08 22:03:36,171 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991536171 2023-04-08 22:03:36,171 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:36,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,172 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,177 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:36,187 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=53819c752f224f75f5ed122f97922705, threads=14 2023-04-08 22:03:36,188 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,188 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,188 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,189 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,189 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,189 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,189 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,190 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,190 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,190 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,194 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,194 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,195 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,197 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,443 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,443 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,446 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,450 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,470 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,478 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,478 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,479 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,489 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,506 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,506 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,510 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,511 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,512 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,513 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 53819c752f224f75f5ed122f97922705 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:36,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:36,554 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:36,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:36,672 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:36,672 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:36,673 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:36,674 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-08 22:03:36,674 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:36,674 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:36,675 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:36,675 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,676 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:36,677 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,677 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:36,677 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,677 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:36,677 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:36,678 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,678 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,678 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:36,678 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:36,679 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:36,679 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:36,680 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:36,680 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991446680 2023-04-08 22:03:36,682 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:36,682 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:36,684 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:36,685 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:36,685 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:36,685 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991476685 2023-04-08 22:03:36,685 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991536685 2023-04-08 22:03:36,685 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:36,695 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=53819c752f224f75f5ed122f97922705, threads=14 2023-04-08 22:03:36,696 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,696 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,696 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,697 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,697 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,697 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,697 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,698 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,698 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,698 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,698 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,699 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,702 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,702 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,833 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,877 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,878 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,882 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,895 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:03:36,898 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,910 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,914 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,930 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,935 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,935 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,938 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,945 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,951 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,953 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 for region: 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:36,953 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 53819c752f224f75f5ed122f97922705 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:37,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:37,011 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/73ae6ed38cae9b85995205c0a1b1aeb0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:37,013 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/85c4a4ed50826801b0ff23921e269eb9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:03:37,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:37,013 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:37,013 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:37,185 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,185 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:37,186 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:37,186 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:37,187 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:37,188 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-08 22:03:37,188 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:37,188 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:37,189 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:37,189 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:37,189 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:37,190 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:37,190 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,190 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:37,190 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,191 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,191 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,191 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:37,192 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,192 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,192 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:37,193 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,193 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:37,193 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:37,195 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:37,195 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,196 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,196 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,196 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,196 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,196 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-04-08 22:03:37,197 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:37,197 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:37,198 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991447198 2023-04-08 22:03:37,200 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,200 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:37,202 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:37,203 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:37,203 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:37,203 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991477203 2023-04-08 22:03:37,203 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991537203 2023-04-08 22:03:37,203 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:37,205 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:37,209 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:37,209 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:37,209 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:37,453 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:37,454 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:37,454 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:37,455 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:37,456 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,457 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:37,458 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,458 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,458 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:37,458 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:37,459 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,459 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,459 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,459 DEBUG [Listener at localhost.localdomain/46567] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,460 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:37,460 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:37,461 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:37,461 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991447461 2023-04-08 22:03:37,463 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:37,463 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,466 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:37,466 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:37,466 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:37,466 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991477466 2023-04-08 22:03:37,466 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991537466 2023-04-08 22:03:37,466 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:37,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:37,482 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:37,482 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:37,482 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:37,483 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:37,717 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,717 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:37,717 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:37,717 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:37,717 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:37,718 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:37,718 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:37,718 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:37,718 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:37,719 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:37,720 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-04-08 22:03:37,720 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:37,720 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:37,720 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:37,721 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:37,721 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:37,721 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:37,721 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,721 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,722 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,723 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:37,723 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,723 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:37,723 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:37,723 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:37,724 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,724 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,724 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,724 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,724 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,724 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:37,725 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:37,725 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:37,725 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991447725 2023-04-08 22:03:37,728 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:37,728 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,730 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:37,731 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:37,731 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:37,731 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991477731 2023-04-08 22:03:37,731 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991537731 2023-04-08 22:03:37,731 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:37,731 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:37,734 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:37,734 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:37,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:37,734 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:37,982 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:37,982 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:37,983 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:37,983 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:37,983 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:37,984 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-08 22:03:37,984 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:37,984 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:37,984 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:37,985 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:37,985 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:37,985 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:37,985 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,985 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:37,986 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:37,987 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,987 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:37,987 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:37,987 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:37,987 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,988 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,988 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,988 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,988 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:37,988 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:37,989 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:37,989 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:37,989 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991447989 2023-04-08 22:03:37,991 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:37,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:37,993 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:37,993 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:37,993 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:37,993 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991477993 2023-04-08 22:03:37,994 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991537994 2023-04-08 22:03:37,994 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:37,994 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:37,996 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:37,996 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:37,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:37,996 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:38,244 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,244 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:38,244 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:38,244 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:38,244 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:38,245 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:38,245 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:38,245 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:38,245 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:38,246 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:38,247 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-08 22:03:38,247 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:38,247 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:38,247 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:38,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:38,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:38,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:38,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:38,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:38,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,250 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,250 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:38,251 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,251 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,251 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,252 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:38,252 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:38,253 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:38,253 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991448253 2023-04-08 22:03:38,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,255 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:38,257 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:38,257 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:38,257 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:38,257 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991478257 2023-04-08 22:03:38,257 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991538257 2023-04-08 22:03:38,258 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:03:38,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:38,262 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN] 2023-04-08 22:03:38,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:38,262 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:38,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:38,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:38,508 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:38,508 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:38,509 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:38,510 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-08 22:03:38,510 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:03:38,510 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:03:38,510 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:38,510 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:38,511 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:38,511 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:38,511 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:38,511 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,511 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,512 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,513 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:38,514 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,514 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,514 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,514 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,514 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-04-08 22:03:38,515 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:38,516 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:38,516 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:38,518 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991448518 2023-04-08 22:03:38,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,519 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:38,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:38,522 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:38,522 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:38,523 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:38,523 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991478523 2023-04-08 22:03:38,523 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991538523 2023-04-08 22:03:38,523 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:38,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:38,528 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:38,528 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:38,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:38,528 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:38,773 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,773 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:38,773 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:38,773 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:38,773 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:38,774 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:38,774 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:38,774 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:38,774 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:38,775 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:38,775 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:03:38,775 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:38,775 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:38,776 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,777 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:38,777 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,777 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:38,777 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,777 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:38,778 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:38,778 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,778 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:38,778 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:38,778 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,778 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,778 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,778 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:38,778 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-04-08 22:03:38,779 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:38,780 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:38,780 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:38,780 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991448780 2023-04-08 22:03:38,781 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:38,782 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:38,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:38,785 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:38,785 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:38,785 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:38,785 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991478785 2023-04-08 22:03:38,785 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991538785 2023-04-08 22:03:38,785 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-08 22:03:38,786 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:38,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:38,788 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:38,788 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:38,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:39,036 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,036 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:39,036 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:39,036 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:39,037 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:39,037 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:39,037 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:39,037 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:39,037 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:39,038 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:39,038 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:39,039 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,040 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,041 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:39,041 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,041 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,041 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:39,041 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,041 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,041 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,041 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:39,041 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-04-08 22:03:39,042 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:39,043 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:39,043 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:39,043 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991449043 2023-04-08 22:03:39,045 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:39,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,046 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,048 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:39,048 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:39,048 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:39,048 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991479048 2023-04-08 22:03:39,049 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991539049 2023-04-08 22:03:39,049 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:39,049 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:39,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:39,052 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-04-08 22:03:39,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:39,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:39,299 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,299 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:39,299 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:39,299 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:39,299 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:39,299 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:39,300 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:39,300 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:39,300 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:39,300 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:39,301 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-08 22:03:39,301 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-04-08 22:03:39,301 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:39,302 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:39,302 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:39,302 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:39,302 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:39,303 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,304 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,304 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:39,304 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,304 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,304 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:39,304 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,305 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,305 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,305 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:03:39,305 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-04-08 22:03:39,305 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:39,306 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:39,306 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:39,306 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991449306 2023-04-08 22:03:39,308 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,308 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:39,311 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,313 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:39,314 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:39,314 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:39,314 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991479314 2023-04-08 22:03:39,314 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991539314 2023-04-08 22:03:39,314 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-08 22:03:39,314 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:03:39,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 exec-time=5.0020 sec 2023-04-08 22:03:39,317 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,318 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,43551,1680991397921; forceNewPlan=false, retain=true 2023-04-08 22:03:39,469 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:39,470 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:39,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991419470"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991419470"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991419470"}]},"ts":"1680991419470"} 2023-04-08 22:03:39,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:39,472 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:03:39,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:03:39,472 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:03:39,564 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:39,565 WARN [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:03:39,565 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:03:39,565 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:03:39,566 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:03:39,567 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-04-08 22:03:39,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:03:39,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:03:39,567 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,568 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,569 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:03:39,569 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,569 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-04-08 22:03:39,570 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:03:39,571 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:03:39,571 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705 to restore RIT 2023-04-08 22:03:39,571 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991449571 2023-04-08 22:03:39,572 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:03:39,572 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:03:39,575 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:03:39,575 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=53819c752f224f75f5ed122f97922705, regionState=OPENING, lastHost=jenkins-hbase20.apache.org,43551,1680991397921, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921, openSeqNum=2 2023-04-08 22:03:39,575 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:03:39,575 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991479575 2023-04-08 22:03:39,575 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991539575 2023-04-08 22:03:39,575 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:03:39,576 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:39,576 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991419576"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991419576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991419576"}]},"ts":"1680991419576"} 2023-04-08 22:03:39,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:39,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:39,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 53819c752f224f75f5ed122f97922705, NAME => 'testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:39,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:39,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:39,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,735 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,736 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1 2023-04-08 22:03:39,736 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1 2023-04-08 22:03:39,737 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53819c752f224f75f5ed122f97922705 columnFamilyName cf1 2023-04-08 22:03:39,744 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 2023-04-08 22:03:39,747 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd 2023-04-08 22:03:39,750 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 2023-04-08 22:03:39,754 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 2023-04-08 22:03:39,758 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 2023-04-08 22:03:39,762 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 2023-04-08 22:03:39,769 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb 2023-04-08 22:03:39,769 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(310): Store=53819c752f224f75f5ed122f97922705/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:39,770 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,771 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2 2023-04-08 22:03:39,771 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2 2023-04-08 22:03:39,771 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 53819c752f224f75f5ed122f97922705 columnFamilyName cf2 2023-04-08 22:03:39,779 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 2023-04-08 22:03:39,783 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b 2023-04-08 22:03:39,787 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 2023-04-08 22:03:39,791 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 2023-04-08 22:03:39,795 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 2023-04-08 22:03:39,801 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e 2023-04-08 22:03:39,805 DEBUG [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 2023-04-08 22:03:39,805 INFO [StoreOpener-53819c752f224f75f5ed122f97922705-1] regionserver.HStore(310): Store=53819c752f224f75f5ed122f97922705/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:39,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,807 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.splits 2023-04-08 22:03:39,810 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:39,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:39,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 53819c752f224f75f5ed122f97922705; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65859293, jitterRate=-0.01862005889415741}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:39,812 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:39,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705., pid=48, masterSystemTime=1680991419730 2023-04-08 22:03:39,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:39,813 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:39,813 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:39,813 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:39,814 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-08 22:03:39,814 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-08 22:03:39,814 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HStore(1912): 53819c752f224f75f5ed122f97922705/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:39,815 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HStore(1912): 53819c752f224f75f5ed122f97922705/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:39,815 INFO [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 53819c752f224f75f5ed122f97922705/cf1 in testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:39,815 INFO [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 53819c752f224f75f5ed122f97922705/cf2 in testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:39,816 INFO [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp, totalSize=34.9 K 2023-04-08 22:03:39,816 INFO [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp, totalSize=34.9 K 2023-04-08 22:03:39,816 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:39,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:39,816 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting 4ddfb947d30944ef8727e64dc8d9ce96, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991412926 2023-04-08 22:03:39,816 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting e412af38f4274eb3abf74cbcf0525f2e, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991412926 2023-04-08 22:03:39,816 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:39,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991419816"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991419816"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991419816"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991419816"}]},"ts":"1680991419816"} 2023-04-08 22:03:39,817 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting e8cd1fa5c3e74cc6b1c16cbe867746b0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991413115 2023-04-08 22:03:39,817 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting a7c10b2f5f5449c1a736d4f1e1488e53, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991413115 2023-04-08 22:03:39,817 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting 86734e03dea04ababbb025727ff6e644, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991413304 2023-04-08 22:03:39,817 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting c59f155a385d43459cef6647013d9023, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991413304 2023-04-08 22:03:39,817 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting f410c0e5508c4af38d0ff2b472aca6cb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991413496 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting 542247275f0a4f16bcce858c72fafa9b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991413496 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting 9a96079839354dd2b7e542ba3c195ad7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991413679 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting 2f3ce244a8624af5b9a99c31da5d98e4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991413679 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting 77a4bee3fd71467d8f0e9a8530729ccd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991413875 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting c4990b55a053410b80d0e47034870f30, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991413875 2023-04-08 22:03:39,818 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] compactions.Compactor(207): Compacting 9b33a90c122d4134be4f3f7ad403b619, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991414050 2023-04-08 22:03:39,819 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] compactions.Compactor(207): Compacting edcfef2570c34079a542d15d61c34580, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991414050 2023-04-08 22:03:39,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-04-08 22:03:39,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 in 240 msec 2023-04-08 22:03:39,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN in 1.5600 sec 2023-04-08 22:03:39,837 INFO [RS:0;jenkins-hbase20:43551-longCompactions-0] throttle.PressureAwareThroughputController(145): 53819c752f224f75f5ed122f97922705#cf2#compaction#50 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:39,837 INFO [RS:0;jenkins-hbase20:43551-shortCompactions-0] throttle.PressureAwareThroughputController(145): 53819c752f224f75f5ed122f97922705#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:39,852 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf1/29077c47226c4c74aee7b840f67cdc91 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/29077c47226c4c74aee7b840f67cdc91 2023-04-08 22:03:39,858 INFO [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 53819c752f224f75f5ed122f97922705/cf1 of 53819c752f224f75f5ed122f97922705 into 29077c47226c4c74aee7b840f67cdc91(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:39,858 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:39,858 INFO [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705., storeName=53819c752f224f75f5ed122f97922705/cf1, priority=9, startTime=1680991419813; duration=0sec 2023-04-08 22:03:39,859 DEBUG [RS:0;jenkins-hbase20:43551-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:39,861 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/.tmp/cf2/63355662e2dc46748945898ea2da5ca5 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/63355662e2dc46748945898ea2da5ca5 2023-04-08 22:03:39,867 INFO [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 53819c752f224f75f5ed122f97922705/cf2 of 53819c752f224f75f5ed122f97922705 into 63355662e2dc46748945898ea2da5ca5(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:39,867 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:39,867 INFO [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705., storeName=53819c752f224f75f5ed122f97922705/cf2, priority=9, startTime=1680991419813; duration=0sec 2023-04-08 22:03:39,867 DEBUG [RS:0;jenkins-hbase20:43551-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:39,927 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-04-08 22:03:39,929 DEBUG [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-04-08 22:03:39,931 INFO [Listener at localhost.localdomain/46567] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-04-08 22:03:39,934 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-04-08 22:03:39,934 INFO [Listener at localhost.localdomain/46567] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-04-08 22:03:39,935 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-04-08 22:03:39,948 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:39,949 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-04-08 22:03:39,950 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackAndDoubleExecution 2023-04-08 22:03:39,951 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:39,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-08 22:03:39,954 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991419954"}]},"ts":"1680991419954"} 2023-04-08 22:03:39,955 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-04-08 22:03:39,956 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-04-08 22:03:39,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN}] 2023-04-08 22:03:39,958 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN 2023-04-08 22:03:39,958 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:39,958 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991419958"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991419958"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991419958"}]},"ts":"1680991419958"} 2023-04-08 22:03:39,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:40,055 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-08 22:03:40,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:40,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 53819c752f224f75f5ed122f97922705, disabling compactions & flushes 2023-04-08 22:03:40,112 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:40,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:40,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. after waiting 0 ms 2023-04-08 22:03:40,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:40,113 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619] to archive 2023-04-08 22:03:40,114 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:40,115 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/4ddfb947d30944ef8727e64dc8d9ce96 2023-04-08 22:03:40,117 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/e8cd1fa5c3e74cc6b1c16cbe867746b0 2023-04-08 22:03:40,118 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/86734e03dea04ababbb025727ff6e644 2023-04-08 22:03:40,119 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/f410c0e5508c4af38d0ff2b472aca6cb 2023-04-08 22:03:40,121 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9a96079839354dd2b7e542ba3c195ad7 2023-04-08 22:03:40,122 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/77a4bee3fd71467d8f0e9a8530729ccd 2023-04-08 22:03:40,123 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/9b33a90c122d4134be4f3f7ad403b619 2023-04-08 22:03:40,128 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580] to archive 2023-04-08 22:03:40,129 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:40,131 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/e412af38f4274eb3abf74cbcf0525f2e 2023-04-08 22:03:40,133 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/a7c10b2f5f5449c1a736d4f1e1488e53 2023-04-08 22:03:40,134 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c59f155a385d43459cef6647013d9023 2023-04-08 22:03:40,136 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/542247275f0a4f16bcce858c72fafa9b 2023-04-08 22:03:40,137 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/2f3ce244a8624af5b9a99c31da5d98e4 2023-04-08 22:03:40,138 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/c4990b55a053410b80d0e47034870f30 2023-04-08 22:03:40,139 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/edcfef2570c34079a542d15d61c34580 2023-04-08 22:03:40,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:03:40,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:40,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705. 2023-04-08 22:03:40,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 53819c752f224f75f5ed122f97922705: 2023-04-08 22:03:40,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 53819c752f224f75f5ed122f97922705 2023-04-08 22:03:40,146 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=53819c752f224f75f5ed122f97922705, regionState=CLOSED 2023-04-08 22:03:40,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991420146"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991420146"}]},"ts":"1680991420146"} 2023-04-08 22:03:40,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-04-08 22:03:40,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 53819c752f224f75f5ed122f97922705, server=jenkins-hbase20.apache.org,43551,1680991397921 in 187 msec 2023-04-08 22:03:40,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-04-08 22:03:40,151 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, UNASSIGN in 192 msec 2023-04-08 22:03:40,152 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991420152"}]},"ts":"1680991420152"} 2023-04-08 22:03:40,153 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-04-08 22:03:40,154 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-04-08 22:03:40,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 205 msec 2023-04-08 22:03:40,256 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-04-08 22:03:40,257 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-04-08 22:03:40,257 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackAndDoubleExecution 2023-04-08 22:03:40,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,260 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-04-08 22:03:40,265 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:40,267 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits] 2023-04-08 22:03:40,269 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/29077c47226c4c74aee7b840f67cdc91 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf1/29077c47226c4c74aee7b840f67cdc91 2023-04-08 22:03:40,271 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/63355662e2dc46748945898ea2da5ca5 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/cf2/63355662e2dc46748945898ea2da5ca5 2023-04-08 22:03:40,274 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705/recovered.edits/90.seqid 2023-04-08 22:03:40,274 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackAndDoubleExecution/53819c752f224f75f5ed122f97922705 2023-04-08 22:03:40,274 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-04-08 22:03:40,278 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,280 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-04-08 22:03:40,281 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-04-08 22:03:40,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-04-08 22:03:40,283 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991420283"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:40,284 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-08 22:03:40,284 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 53819c752f224f75f5ed122f97922705, NAME => 'testRollbackAndDoubleExecution,,1680991412418.53819c752f224f75f5ed122f97922705.', STARTKEY => '', ENDKEY => ''}] 2023-04-08 22:03:40,284 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-04-08 22:03:40,284 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991420284"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:40,287 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-04-08 22:03:40,288 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:03:40,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 30 msec 2023-04-08 22:03:40,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-04-08 22:03:40,363 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-04-08 22:03:40,378 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400 (was 396) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48886 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:41490 [Waiting for operation #15] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:42464 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:42128 [Waiting for operation #46] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:50786 [Waiting for operation #33] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48862 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48848 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase20:43551-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:42060 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48868 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:42110 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48054 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:42144 [Waiting for operation #47] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48870 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:41474 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:48896 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-3-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-660310859_15 at /127.0.0.1:57992 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=254 (was 225) - SystemLoadAverage LEAK? -, ProcessCount=165 (was 165), AvailableMemoryMB=4175 (was 4194) 2023-04-08 22:03:40,392 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=400, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=254, ProcessCount=165, AvailableMemoryMB=4174 2023-04-08 22:03:40,393 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:40,394 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:40,395 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:40,396 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:40,398 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:40,399 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:40,400 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,401 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 empty. 2023-04-08 22:03:40,401 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,401 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-04-08 22:03:40,411 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:40,412 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 18826054628d89f6df3d6e684f01e947, NAME => 'testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 18826054628d89f6df3d6e684f01e947, disabling compactions & flushes 2023-04-08 22:03:40,422 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. after waiting 0 ms 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,422 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,422 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 18826054628d89f6df3d6e684f01e947: 2023-04-08 22:03:40,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:40,425 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991420425"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991420425"}]},"ts":"1680991420425"} 2023-04-08 22:03:40,426 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:40,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:40,427 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991420427"}]},"ts":"1680991420427"} 2023-04-08 22:03:40,429 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-04-08 22:03:40,431 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:40,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:40,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:40,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:40,432 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:40,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, ASSIGN}] 2023-04-08 22:03:40,434 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, ASSIGN 2023-04-08 22:03:40,434 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:40,585 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:40,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=18826054628d89f6df3d6e684f01e947, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:40,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991420586"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991420586"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991420586"}]},"ts":"1680991420586"} 2023-04-08 22:03:40,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 18826054628d89f6df3d6e684f01e947, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:40,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 18826054628d89f6df3d6e684f01e947, NAME => 'testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:40,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:40,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:40,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,743 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,744 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,745 DEBUG [StoreOpener-18826054628d89f6df3d6e684f01e947-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf1 2023-04-08 22:03:40,745 DEBUG [StoreOpener-18826054628d89f6df3d6e684f01e947-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf1 2023-04-08 22:03:40,746 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18826054628d89f6df3d6e684f01e947 columnFamilyName cf1 2023-04-08 22:03:40,746 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] regionserver.HStore(310): Store=18826054628d89f6df3d6e684f01e947/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:40,747 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,748 DEBUG [StoreOpener-18826054628d89f6df3d6e684f01e947-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf2 2023-04-08 22:03:40,748 DEBUG [StoreOpener-18826054628d89f6df3d6e684f01e947-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf2 2023-04-08 22:03:40,748 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18826054628d89f6df3d6e684f01e947 columnFamilyName cf2 2023-04-08 22:03:40,749 INFO [StoreOpener-18826054628d89f6df3d6e684f01e947-1] regionserver.HStore(310): Store=18826054628d89f6df3d6e684f01e947/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:40,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:40,753 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:40,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:40,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 18826054628d89f6df3d6e684f01e947; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74963965, jitterRate=0.11705012619495392}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:40,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 18826054628d89f6df3d6e684f01e947: 2023-04-08 22:03:40,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947., pid=55, masterSystemTime=1680991420739 2023-04-08 22:03:40,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:40,759 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=18826054628d89f6df3d6e684f01e947, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:40,759 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991420759"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991420759"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991420759"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991420759"}]},"ts":"1680991420759"} 2023-04-08 22:03:40,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-04-08 22:03:40,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 18826054628d89f6df3d6e684f01e947, server=jenkins-hbase20.apache.org,41749,1680991397967 in 172 msec 2023-04-08 22:03:40,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-04-08 22:03:40,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, ASSIGN in 330 msec 2023-04-08 22:03:40,764 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:40,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991420764"}]},"ts":"1680991420764"} 2023-04-08 22:03:40,765 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-04-08 22:03:40,767 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:40,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 372 msec 2023-04-08 22:03:40,897 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:40,899 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:40,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:40,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:40,900 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:40,902 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, UNASSIGN}] 2023-04-08 22:03:40,903 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, UNASSIGN 2023-04-08 22:03:40,903 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=18826054628d89f6df3d6e684f01e947, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:40,903 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991420903"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991420903"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991420903"}]},"ts":"1680991420903"} 2023-04-08 22:03:40,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 18826054628d89f6df3d6e684f01e947, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:41,057 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:41,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 18826054628d89f6df3d6e684f01e947, disabling compactions & flushes 2023-04-08 22:03:41,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:41,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:41,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. after waiting 0 ms 2023-04-08 22:03:41,058 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:41,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-04-08 22:03:41,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:41,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947. 2023-04-08 22:03:41,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 18826054628d89f6df3d6e684f01e947: 2023-04-08 22:03:41,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:41,065 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=18826054628d89f6df3d6e684f01e947, regionState=CLOSED 2023-04-08 22:03:41,065 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991421065"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991421065"}]},"ts":"1680991421065"} 2023-04-08 22:03:41,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-04-08 22:03:41,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 18826054628d89f6df3d6e684f01e947, server=jenkins-hbase20.apache.org,41749,1680991397967 in 162 msec 2023-04-08 22:03:41,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-04-08 22:03:41,069 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=18826054628d89f6df3d6e684f01e947, UNASSIGN in 166 msec 2023-04-08 22:03:41,106 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-04-08 22:03:41,108 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-04-08 22:03:41,110 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1680991421110"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1680991421110"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1680991421110"}]},"ts":"1680991421110"} 2023-04-08 22:03:41,110 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421110"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991421110"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991421110"}]},"ts":"1680991421110"} 2023-04-08 22:03:41,111 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421110"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991421110"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991421110"}]},"ts":"1680991421110"} 2023-04-08 22:03:41,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, ASSIGN}] 2023-04-08 22:03:41,115 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, ASSIGN 2023-04-08 22:03:41,116 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:41,116 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, ASSIGN 2023-04-08 22:03:41,117 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:41,266 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:03:41,267 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=861bb60d37055f19de7791db1f5f8627, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,268 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421267"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991421267"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991421267"}]},"ts":"1680991421267"} 2023-04-08 22:03:41,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 861bb60d37055f19de7791db1f5f8627, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:41,271 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=cb76bcec70ebbefd8b284e846da96f4e, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,271 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421271"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991421271"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991421271"}]},"ts":"1680991421271"} 2023-04-08 22:03:41,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure cb76bcec70ebbefd8b284e846da96f4e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:41,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cb76bcec70ebbefd8b284e846da96f4e, NAME => 'testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.', STARTKEY => '41', ENDKEY => ''} 2023-04-08 22:03:41,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,424 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,426 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,427 DEBUG [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf1 2023-04-08 22:03:41,427 DEBUG [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf1 2023-04-08 22:03:41,428 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb76bcec70ebbefd8b284e846da96f4e columnFamilyName cf1 2023-04-08 22:03:41,428 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] regionserver.HStore(310): Store=cb76bcec70ebbefd8b284e846da96f4e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:41,428 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,430 DEBUG [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf2 2023-04-08 22:03:41,430 DEBUG [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf2 2023-04-08 22:03:41,430 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cb76bcec70ebbefd8b284e846da96f4e columnFamilyName cf2 2023-04-08 22:03:41,431 INFO [StoreOpener-cb76bcec70ebbefd8b284e846da96f4e-1] regionserver.HStore(310): Store=cb76bcec70ebbefd8b284e846da96f4e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:41,432 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,433 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,435 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:41,436 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened cb76bcec70ebbefd8b284e846da96f4e; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61659647, jitterRate=-0.08119966089725494}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:41,437 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for cb76bcec70ebbefd8b284e846da96f4e: 2023-04-08 22:03:41,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e., pid=62, masterSystemTime=1680991421421 2023-04-08 22:03:41,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,439 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 861bb60d37055f19de7791db1f5f8627, NAME => 'testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.', STARTKEY => '', ENDKEY => '41'} 2023-04-08 22:03:41,439 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=cb76bcec70ebbefd8b284e846da96f4e, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,439 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421439"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991421439"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991421439"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991421439"}]},"ts":"1680991421439"} 2023-04-08 22:03:41,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,440 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,441 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-04-08 22:03:41,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure cb76bcec70ebbefd8b284e846da96f4e, server=jenkins-hbase20.apache.org,41749,1680991397967 in 168 msec 2023-04-08 22:03:41,443 DEBUG [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf1 2023-04-08 22:03:41,443 DEBUG [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf1 2023-04-08 22:03:41,443 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 861bb60d37055f19de7791db1f5f8627 columnFamilyName cf1 2023-04-08 22:03:41,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, ASSIGN in 328 msec 2023-04-08 22:03:41,443 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] regionserver.HStore(310): Store=861bb60d37055f19de7791db1f5f8627/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:41,444 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,445 DEBUG [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf2 2023-04-08 22:03:41,445 DEBUG [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf2 2023-04-08 22:03:41,445 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 861bb60d37055f19de7791db1f5f8627 columnFamilyName cf2 2023-04-08 22:03:41,445 INFO [StoreOpener-861bb60d37055f19de7791db1f5f8627-1] regionserver.HStore(310): Store=861bb60d37055f19de7791db1f5f8627/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:41,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,447 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,448 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:41,450 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 861bb60d37055f19de7791db1f5f8627; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65153590, jitterRate=-0.029135853052139282}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:41,451 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 861bb60d37055f19de7791db1f5f8627: 2023-04-08 22:03:41,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627., pid=61, masterSystemTime=1680991421421 2023-04-08 22:03:41,453 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,453 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=861bb60d37055f19de7791db1f5f8627, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,453 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421453"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991421453"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991421453"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991421453"}]},"ts":"1680991421453"} 2023-04-08 22:03:41,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-04-08 22:03:41,456 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 861bb60d37055f19de7791db1f5f8627, server=jenkins-hbase20.apache.org,41749,1680991397967 in 186 msec 2023-04-08 22:03:41,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-04-08 22:03:41,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, ASSIGN in 342 msec 2023-04-08 22:03:41,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e in 560 msec 2023-04-08 22:03:41,653 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:03:41,656 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:41,658 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,658 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,659 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,662 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-08 22:03:41,662 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991421662"}]},"ts":"1680991421662"} 2023-04-08 22:03:41,663 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-04-08 22:03:41,665 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-04-08 22:03:41,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, UNASSIGN}] 2023-04-08 22:03:41,666 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, UNASSIGN 2023-04-08 22:03:41,667 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=cb76bcec70ebbefd8b284e846da96f4e, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,667 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421667"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991421667"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991421667"}]},"ts":"1680991421667"} 2023-04-08 22:03:41,668 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure cb76bcec70ebbefd8b284e846da96f4e, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:41,671 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, UNASSIGN 2023-04-08 22:03:41,671 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=861bb60d37055f19de7791db1f5f8627, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:41,671 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421671"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991421671"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991421671"}]},"ts":"1680991421671"} 2023-04-08 22:03:41,672 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 861bb60d37055f19de7791db1f5f8627, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:41,763 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-08 22:03:41,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing cb76bcec70ebbefd8b284e846da96f4e, disabling compactions & flushes 2023-04-08 22:03:41,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. after waiting 0 ms 2023-04-08 22:03:41,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,825 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-08 22:03:41,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:41,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e. 2023-04-08 22:03:41,826 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for cb76bcec70ebbefd8b284e846da96f4e: 2023-04-08 22:03:41,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,828 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 861bb60d37055f19de7791db1f5f8627, disabling compactions & flushes 2023-04-08 22:03:41,829 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. after waiting 0 ms 2023-04-08 22:03:41,829 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,829 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=cb76bcec70ebbefd8b284e846da96f4e, regionState=CLOSED 2023-04-08 22:03:41,829 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991421829"}]},"ts":"1680991421829"} 2023-04-08 22:03:41,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-04-08 22:03:41,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure cb76bcec70ebbefd8b284e846da96f4e, server=jenkins-hbase20.apache.org,41749,1680991397967 in 162 msec 2023-04-08 22:03:41,832 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-08 22:03:41,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:41,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627. 2023-04-08 22:03:41,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 861bb60d37055f19de7791db1f5f8627: 2023-04-08 22:03:41,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=cb76bcec70ebbefd8b284e846da96f4e, UNASSIGN in 167 msec 2023-04-08 22:03:41,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,835 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=861bb60d37055f19de7791db1f5f8627, regionState=CLOSED 2023-04-08 22:03:41,835 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991421835"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991421835"}]},"ts":"1680991421835"} 2023-04-08 22:03:41,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-04-08 22:03:41,837 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 861bb60d37055f19de7791db1f5f8627, server=jenkins-hbase20.apache.org,41749,1680991397967 in 164 msec 2023-04-08 22:03:41,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-04-08 22:03:41,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=861bb60d37055f19de7791db1f5f8627, UNASSIGN in 173 msec 2023-04-08 22:03:41,840 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991421839"}]},"ts":"1680991421839"} 2023-04-08 22:03:41,841 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-04-08 22:03:41,842 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-04-08 22:03:41,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 185 msec 2023-04-08 22:03:41,964 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-04-08 22:03:41,964 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-04-08 22:03:41,965 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,968 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,969 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,969 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-04-08 22:03:41,973 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:41,973 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,973 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,975 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/recovered.edits] 2023-04-08 22:03:41,975 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/recovered.edits] 2023-04-08 22:03:41,975 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/recovered.edits] 2023-04-08 22:03:41,982 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/recovered.edits/7.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627/recovered.edits/7.seqid 2023-04-08 22:03:41,982 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/recovered.edits/7.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e/recovered.edits/7.seqid 2023-04-08 22:03:41,982 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/recovered.edits/4.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947/recovered.edits/4.seqid 2023-04-08 22:03:41,983 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/861bb60d37055f19de7791db1f5f8627 2023-04-08 22:03:41,983 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:03:41,983 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionNoStoreFile/18826054628d89f6df3d6e684f01e947 2023-04-08 22:03:41,983 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-04-08 22:03:41,985 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,987 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-04-08 22:03:41,989 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-04-08 22:03:41,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-04-08 22:03:41,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991421990"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:41,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991421990"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:41,990 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991421990"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:41,992 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:03:41,992 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 18826054628d89f6df3d6e684f01e947, NAME => 'testSplitTableRegionNoStoreFile,,1680991420395.18826054628d89f6df3d6e684f01e947.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 861bb60d37055f19de7791db1f5f8627, NAME => 'testSplitTableRegionNoStoreFile,,1680991420898.861bb60d37055f19de7791db1f5f8627.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => cb76bcec70ebbefd8b284e846da96f4e, NAME => 'testSplitTableRegionNoStoreFile,41,1680991420898.cb76bcec70ebbefd8b284e846da96f4e.', STARTKEY => '41', ENDKEY => ''}] 2023-04-08 22:03:41,992 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-04-08 22:03:41,993 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991421992"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:41,994 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-04-08 22:03:41,996 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:03:41,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 31 msec 2023-04-08 22:03:42,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-04-08 22:03:42,070 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-04-08 22:03:42,088 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=404 (was 400) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1774230861_15 at /127.0.0.1:48862 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1774230861_15 at /127.0.0.1:41474 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=250 (was 254), ProcessCount=165 (was 165), AvailableMemoryMB=4170 (was 4174) 2023-04-08 22:03:42,106 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=404, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=250, ProcessCount=165, AvailableMemoryMB=4169 2023-04-08 22:03:42,106 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:42,108 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:42,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:42,110 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,113 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:42,113 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:42,115 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,116 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a empty. 2023-04-08 22:03:42,116 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,116 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-04-08 22:03:42,134 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:42,135 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => ee9b8d200835a543c2b4c8142e56c58a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:42,144 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:42,144 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing ee9b8d200835a543c2b4c8142e56c58a, disabling compactions & flushes 2023-04-08 22:03:42,144 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,144 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,145 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. after waiting 0 ms 2023-04-08 22:03:42,145 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,145 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,145 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:42,147 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:42,148 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991422148"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991422148"}]},"ts":"1680991422148"} 2023-04-08 22:03:42,149 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:42,150 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:42,150 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991422150"}]},"ts":"1680991422150"} 2023-04-08 22:03:42,152 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-04-08 22:03:42,153 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:42,154 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:42,154 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:42,154 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:42,155 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:42,155 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, ASSIGN}] 2023-04-08 22:03:42,156 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, ASSIGN 2023-04-08 22:03:42,157 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:03:42,307 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:42,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ee9b8d200835a543c2b4c8142e56c58a, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991422309"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991422309"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991422309"}]},"ts":"1680991422309"} 2023-04-08 22:03:42,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure ee9b8d200835a543c2b4c8142e56c58a, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:42,464 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,464 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-08 22:03:42,468 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:32976, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-08 22:03:42,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ee9b8d200835a543c2b4c8142e56c58a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:42,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:42,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:42,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,476 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,477 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,479 DEBUG [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1 2023-04-08 22:03:42,479 DEBUG [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1 2023-04-08 22:03:42,479 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee9b8d200835a543c2b4c8142e56c58a columnFamilyName cf1 2023-04-08 22:03:42,480 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] regionserver.HStore(310): Store=ee9b8d200835a543c2b4c8142e56c58a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:42,480 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,482 DEBUG [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2 2023-04-08 22:03:42,482 DEBUG [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2 2023-04-08 22:03:42,482 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ee9b8d200835a543c2b4c8142e56c58a columnFamilyName cf2 2023-04-08 22:03:42,483 INFO [StoreOpener-ee9b8d200835a543c2b4c8142e56c58a-1] regionserver.HStore(310): Store=ee9b8d200835a543c2b4c8142e56c58a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:42,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,484 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,486 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:42,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:42,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:42,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened ee9b8d200835a543c2b4c8142e56c58a; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61986107, jitterRate=-0.07633502781391144}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:42,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:42,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a., pid=71, masterSystemTime=1680991422464 2023-04-08 22:03:42,501 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,503 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ee9b8d200835a543c2b4c8142e56c58a, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,503 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991422503"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991422503"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991422503"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991422503"}]},"ts":"1680991422503"} 2023-04-08 22:03:42,506 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-04-08 22:03:42,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure ee9b8d200835a543c2b4c8142e56c58a, server=jenkins-hbase20.apache.org,41779,1680991398002 in 194 msec 2023-04-08 22:03:42,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-04-08 22:03:42,508 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, ASSIGN in 352 msec 2023-04-08 22:03:42,509 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:42,509 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991422509"}]},"ts":"1680991422509"} 2023-04-08 22:03:42,511 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-04-08 22:03:42,512 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:42,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 403 msec 2023-04-08 22:03:42,613 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-04-08 22:03:42,616 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:32990, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-04-08 22:03:42,623 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:42,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,625 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:42,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:42,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:42,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,626 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,627 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,627 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,627 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,629 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,629 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,629 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,629 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:42,629 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:42,629 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,629 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:42,630 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:03:42,630 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,641 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/ee19d9150cfd43768dcdb9d44c50df89 2023-04-08 22:03:42,661 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb 2023-04-08 22:03:42,665 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/ee19d9150cfd43768dcdb9d44c50df89 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89 2023-04-08 22:03:42,670 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:42,671 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb 2023-04-08 22:03:42,676 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:03:42,677 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 47ms, sequenceid=6, compaction requested=false 2023-04-08 22:03:42,677 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:42,678 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:42,678 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,678 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:42,678 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:42,678 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,678 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,678 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,679 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,679 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:42,682 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@63bd8e56[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:42,682 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:42,682 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,683 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:42,683 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:42,684 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,684 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,684 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,684 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,684 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,685 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,685 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,685 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,685 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,685 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:42,687 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:42,687 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:42,687 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:42,687 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:42,687 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,687 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:42,688 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,690 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,690 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,692 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,692 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,692 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:42,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:42,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:42,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,700 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,800 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:42,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:42,817 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:42,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:42,819 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,819 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:42,819 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,819 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:42,819 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:42,819 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,819 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,820 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,820 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,820 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,820 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,821 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,821 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,821 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:42,822 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,822 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,822 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,823 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,824 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,824 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,824 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,824 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,824 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,830 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:42,830 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,830 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:42,830 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:42,830 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:42,844 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/4e79ace01e9a42f5aac7f0ccb078f238 2023-04-08 22:03:42,860 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/7e2e1210cb8b45cdb30080e19244ed20 2023-04-08 22:03:42,864 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/4e79ace01e9a42f5aac7f0ccb078f238 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238 2023-04-08 22:03:42,868 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:42,869 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/7e2e1210cb8b45cdb30080e19244ed20 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20 2023-04-08 22:03:42,873 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:03:42,873 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 43ms, sequenceid=19, compaction requested=false 2023-04-08 22:03:42,874 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:42,874 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:42,874 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:42,874 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:42,874 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,874 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:42,874 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:42,875 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,875 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,875 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,875 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:42,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:42,877 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1bb682d1[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:42,877 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:42,877 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,877 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,877 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,877 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,878 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,878 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:42,878 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,879 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,879 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:42,882 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:42,882 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:42,882 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:42,882 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:42,882 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,882 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:42,884 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:42,884 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,884 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,885 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,886 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:42,886 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,887 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,888 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,889 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:42,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:42,889 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:42,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:42,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:42,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:42,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,889 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,889 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,890 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:42,890 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:42,890 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:42,890 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,890 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:42,990 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:42,991 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,007 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,008 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,009 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,009 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,010 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,010 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,011 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,011 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,011 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,011 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,011 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:43,013 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,013 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,014 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,014 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,014 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,014 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,021 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/6a96fe0f504c475b88397a07c0c43291 2023-04-08 22:03:43,032 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/742ac42086de40749f2c35421eaab183 2023-04-08 22:03:43,037 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/6a96fe0f504c475b88397a07c0c43291 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291 2023-04-08 22:03:43,041 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:43,041 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/742ac42086de40749f2c35421eaab183 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183 2023-04-08 22:03:43,046 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:03:43,047 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 36ms, sequenceid=32, compaction requested=true 2023-04-08 22:03:43,047 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,047 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,047 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,047 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,047 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,047 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,047 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,048 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,048 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,049 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,049 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,050 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5015ad06[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,050 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,050 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,051 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,051 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,052 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,052 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,053 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,053 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,053 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,055 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,055 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,055 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,055 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,055 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,055 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,056 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,056 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,057 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,057 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,058 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,059 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,059 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,059 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,059 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,061 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:43,061 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:43,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,061 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,062 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,162 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:43,163 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,181 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,181 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,181 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,181 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,182 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,182 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,182 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,182 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,182 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,182 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,183 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,183 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,184 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,184 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,184 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,184 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,185 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:43,185 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,186 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,186 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,197 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/f19a6beb8b0b46d8be9bc508884380fa 2023-04-08 22:03:43,213 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/207585c27ccc4a0b8b27a60ba79c3640 2023-04-08 22:03:43,217 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/f19a6beb8b0b46d8be9bc508884380fa as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa 2023-04-08 22:03:43,220 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:43,221 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/207585c27ccc4a0b8b27a60ba79c3640 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640 2023-04-08 22:03:43,226 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:03:43,227 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 42ms, sequenceid=45, compaction requested=true 2023-04-08 22:03:43,227 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,227 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,227 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,227 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,227 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,227 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,227 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,228 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,228 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,229 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,229 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7217f32a[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,230 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,231 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,231 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,231 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,232 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,232 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,233 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,236 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,236 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,236 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,236 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,239 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,239 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,239 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,239 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,239 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,239 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,240 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,243 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,243 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,243 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,245 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:43,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:43,245 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:43,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,346 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:43,347 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,359 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,360 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,361 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,361 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,361 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,361 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,361 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,361 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,362 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,362 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,362 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,363 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,363 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,364 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,364 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,364 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,364 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,364 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,364 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,364 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,364 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,365 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,385 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/c08744d7e7dc403c9666e129ac13cee3 2023-04-08 22:03:43,401 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/1f1871ccd13f4ac281e1a84c49580da9 2023-04-08 22:03:43,405 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/c08744d7e7dc403c9666e129ac13cee3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3 2023-04-08 22:03:43,410 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:43,411 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/1f1871ccd13f4ac281e1a84c49580da9 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9 2023-04-08 22:03:43,415 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:03:43,416 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 51ms, sequenceid=58, compaction requested=true 2023-04-08 22:03:43,416 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,416 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,416 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,416 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,416 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,416 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,416 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,417 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,417 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,417 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,417 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,419 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d570c00[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,419 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,419 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,420 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,420 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,420 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,421 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,421 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,421 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,422 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,424 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,424 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,424 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,424 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,424 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,424 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,425 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,425 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,425 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,425 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,425 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,426 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,427 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,429 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,429 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,429 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,440 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,440 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,441 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,541 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:43,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,566 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,568 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,568 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,568 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,568 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,568 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,569 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,569 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,570 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,572 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,572 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,572 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,572 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,573 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,573 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,573 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,573 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,573 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,574 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,584 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/656c9bbb21614516a9ee1e82205cf07c 2023-04-08 22:03:43,603 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/69b8350e1dd6467ba3fc208e2f408cbe 2023-04-08 22:03:43,610 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/656c9bbb21614516a9ee1e82205cf07c as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c 2023-04-08 22:03:43,616 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:43,616 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/69b8350e1dd6467ba3fc208e2f408cbe as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe 2023-04-08 22:03:43,620 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:03:43,621 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 48ms, sequenceid=71, compaction requested=true 2023-04-08 22:03:43,621 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,622 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,622 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,622 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,622 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,622 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,622 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,623 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,623 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,623 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,623 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,625 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5258b971[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,625 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,626 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,626 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,626 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,627 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,628 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,628 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,632 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,632 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,632 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,632 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,632 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,634 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,633 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,634 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,634 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,636 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,636 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,636 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,638 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,638 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:43,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,638 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,639 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,639 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:43,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,639 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:43,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,639 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,639 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,640 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,640 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,640 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,739 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:43,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,764 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,765 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,765 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,766 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,766 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,766 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,766 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,767 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,768 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,768 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,768 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,769 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,769 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,769 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-04-08 22:03:43,769 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,770 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,771 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,771 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,771 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,771 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,771 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,781 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:43,784 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:43,792 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:43,795 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:43,796 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/63bf6ad8d3a94c5e89847f18667fefc0 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:43,800 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:43,800 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0, entries=9, sequenceid=83, filesize=5.1 K 2023-04-08 22:03:43,801 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/9dc1af50a9604610802cf1b540036654 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:43,805 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:43,805 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654, entries=9, sequenceid=83, filesize=5.1 K 2023-04-08 22:03:43,806 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 37ms, sequenceid=83, compaction requested=true 2023-04-08 22:03:43,806 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,806 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,806 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,806 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,806 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,806 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,806 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,808 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,808 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,808 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,808 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,810 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f5853a1[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,810 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,810 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,811 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,811 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,811 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,811 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,811 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,812 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,812 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,812 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,812 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,812 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,813 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,813 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,814 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,818 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,818 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,818 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,818 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,818 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,823 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,824 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,824 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,824 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,825 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,826 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,827 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,827 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,829 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,829 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,830 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,830 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,830 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,830 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,831 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,831 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,831 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:43,831 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,831 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,932 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:43,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:43,945 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:43,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:43,946 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,946 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:43,946 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,947 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:43,947 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:43,947 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,947 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,948 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,948 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,949 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:43,949 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,950 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:43,950 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:43,950 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,950 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,950 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:43,950 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-04-08 22:03:43,950 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,951 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,951 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,951 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,951 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,951 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,959 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:43,962 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:43,969 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:43,972 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:43,972 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/0eae0cd2866a4f71a78e067c3b866524 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:43,976 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:43,976 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524, entries=5, sequenceid=91, filesize=5.0 K 2023-04-08 22:03:43,977 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/b32cbf9dfdf040ab9a1006917be8c630 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:43,981 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:43,981 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630, entries=5, sequenceid=91, filesize=5.0 K 2023-04-08 22:03:43,982 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 32ms, sequenceid=91, compaction requested=true 2023-04-08 22:03:43,982 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:43,982 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:43,982 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:43,982 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:43,982 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,982 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:43,982 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:43,983 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,984 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,984 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,984 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:43,986 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@621b7bc3[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:43,986 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:43,987 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,988 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,988 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,988 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:43,988 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:43,989 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,989 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:43,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:43,994 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:43,994 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:43,994 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:43,994 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,994 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:43,997 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:43,998 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,998 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:43,999 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,001 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:44,001 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,001 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,010 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,010 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,009 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:44,010 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:44,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,009 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,009 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:44,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,010 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,011 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,012 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,111 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:44,114 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:44,132 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:03:44,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:03:44,134 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,134 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:44,134 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:44,134 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-04-08 22:03:44,134 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-04-08 22:03:44,135 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,135 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,136 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,136 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,136 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,136 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,137 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,137 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:03:44,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:44,138 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:44,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-04-08 22:03:44,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:44,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:03:44,139 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:03:44,139 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,139 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. started... 2023-04-08 22:03:44,139 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,140 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:44,147 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,151 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,160 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,165 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,166 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/1088cfa683174ff7a7f67d0ca4e26705 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,171 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,171 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705, entries=5, sequenceid=99, filesize=5.0 K 2023-04-08 22:03:44,172 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,177 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,178 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09, entries=5, sequenceid=99, filesize=5.0 K 2023-04-08 22:03:44,179 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 40ms, sequenceid=99, compaction requested=true 2023-04-08 22:03:44,179 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:44,179 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,179 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:03:44,179 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:03:44,179 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,179 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-04-08 22:03:44,179 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-04-08 22:03:44,180 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,180 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:44,181 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,181 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:03:44,181 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:44,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-04-08 22:03:44,184 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@27695c8c[Count = 0] remaining members to acquire global barrier 2023-04-08 22:03:44,184 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-04-08 22:03:44,184 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,184 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,184 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,184 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,185 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:03:44,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-04-08 22:03:44,185 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:03:44,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:44,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:44,186 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:44,187 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:44,187 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-04-08 22:03:44,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:44,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:44,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:44,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:03:44,191 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:03:44,191 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-04-08 22:03:44,191 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:03:44,191 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:03:44,191 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,191 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:03:44,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:03:44,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,194 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:03:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:03:44,195 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,195 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,195 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,197 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-04-08 22:03:44,198 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:03:44,197 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:03:44,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:03:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,197 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:03:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:03:44,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:03:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,198 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:44,198 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:03:44,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:03:44,199 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,199 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:44,298 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:03:44,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:03:44,306 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,308 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:44,310 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:44,310 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:44,310 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:44,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, UNASSIGN}] 2023-04-08 22:03:44,314 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, UNASSIGN 2023-04-08 22:03:44,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ee9b8d200835a543c2b4c8142e56c58a, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991424315"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991424315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991424315"}]},"ts":"1680991424315"} 2023-04-08 22:03:44,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure ee9b8d200835a543c2b4c8142e56c58a, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:44,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing ee9b8d200835a543c2b4c8142e56c58a, disabling compactions & flushes 2023-04-08 22:03:44,470 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,470 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,471 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. after waiting 0 ms 2023-04-08 22:03:44,471 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,471 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing ee9b8d200835a543c2b4c8142e56c58a 2/2 column families, dataSize=50 B heapSize=720 B 2023-04-08 22:03:44,487 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,505 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,511 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,512 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf1/cc40963b18f24df6b366ab36b62fc605 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,519 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,519 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605, entries=1, sequenceid=103, filesize=4.9 K 2023-04-08 22:03:44,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.tmp/cf2/0db9de72723849b4b02b8b38643f65c3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3, entries=1, sequenceid=103, filesize=4.9 K 2023-04-08 22:03:44,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ee9b8d200835a543c2b4c8142e56c58a in 56ms, sequenceid=103, compaction requested=true 2023-04-08 22:03:44,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-04-08 22:03:44,534 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:44,535 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a. 2023-04-08 22:03:44,535 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for ee9b8d200835a543c2b4c8142e56c58a: 2023-04-08 22:03:44,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,537 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ee9b8d200835a543c2b4c8142e56c58a, regionState=CLOSED 2023-04-08 22:03:44,537 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991424536"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991424536"}]},"ts":"1680991424536"} 2023-04-08 22:03:44,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-04-08 22:03:44,539 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure ee9b8d200835a543c2b4c8142e56c58a, server=jenkins-hbase20.apache.org,41779,1680991398002 in 221 msec 2023-04-08 22:03:44,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-04-08 22:03:44,541 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ee9b8d200835a543c2b4c8142e56c58a, UNASSIGN in 226 msec 2023-04-08 22:03:44,557 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=ee9b8d200835a543c2b4c8142e56c58a, threads=16 2023-04-08 22:03:44,558 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,558 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,558 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,558 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,558 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,559 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,562 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,562 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,562 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,562 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,566 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,587 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,587 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,587 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,588 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,589 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,591 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9, top=false 2023-04-08 22:03:44,593 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,594 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe, top=false 2023-04-08 22:03:44,594 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:44,596 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:44,596 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640, top=false 2023-04-08 22:03:44,598 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c, top=false 2023-04-08 22:03:44,601 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:44,602 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,603 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,604 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:44,604 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3, top=true 2023-04-08 22:03:44,607 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:44,607 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:44,607 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524, top=true 2023-04-08 22:03:44,623 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:44,623 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09, top=true 2023-04-08 22:03:44,623 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:44,623 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630, top=true 2023-04-08 22:03:44,624 INFO [StoreFileSplitter-pool-11] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:44,624 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705, top=true 2023-04-08 22:03:44,630 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9 for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,632 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,638 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,642 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640 for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,645 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,646 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,650 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,656 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,656 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,656 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,657 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,657 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,659 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3, top=false 2023-04-08 22:03:44,659 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa, top=false 2023-04-08 22:03:44,659 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,663 INFO [StoreFileSplitter-pool-10] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:44,663 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605, top=true 2023-04-08 22:03:44,670 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,670 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,678 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605 for child: ae1a4e6d25e5fb5dd5133abebab5ecb7, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,678 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,687 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,687 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/.splits/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3 for child: cfac3b4c98d7f2cc5feb51a9996121dc, parent: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,690 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,690 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,690 INFO [StoreFileSplitter-pool-7] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:44,691 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,692 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:44,694 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,694 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,697 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,706 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,712 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,713 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,718 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,722 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,722 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,724 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89 for region: ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:44,724 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region ee9b8d200835a543c2b4c8142e56c58a Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:03:44,763 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-04-08 22:03:44,765 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-04-08 22:03:44,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1680991424766"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1680991424766"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1680991424766"}]},"ts":"1680991424766"} 2023-04-08 22:03:44,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991424766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991424766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991424766"}]},"ts":"1680991424766"} 2023-04-08 22:03:44,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991424766"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991424766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991424766"}]},"ts":"1680991424766"} 2023-04-08 22:03:44,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, ASSIGN}] 2023-04-08 22:03:44,770 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, ASSIGN 2023-04-08 22:03:44,770 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:03:44,771 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, ASSIGN 2023-04-08 22:03:44,771 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:03:44,921 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:03:44,922 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=cfac3b4c98d7f2cc5feb51a9996121dc, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,922 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991424922"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991424922"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991424922"}]},"ts":"1680991424922"} 2023-04-08 22:03:44,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure cfac3b4c98d7f2cc5feb51a9996121dc, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:44,927 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ae1a4e6d25e5fb5dd5133abebab5ecb7, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:44,927 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991424926"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991424926"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991424926"}]},"ts":"1680991424926"} 2023-04-08 22:03:44,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure ae1a4e6d25e5fb5dd5133abebab5ecb7, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:45,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,083 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => cfac3b4c98d7f2cc5feb51a9996121dc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.', STARTKEY => '', ENDKEY => '60'} 2023-04-08 22:03:45,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:45,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:45,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,084 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,088 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,090 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1 2023-04-08 22:03:45,091 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1 2023-04-08 22:03:45,091 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfac3b4c98d7f2cc5feb51a9996121dc columnFamilyName cf1 2023-04-08 22:03:45,108 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-bottom 2023-04-08 22:03:45,112 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,114 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-bottom 2023-04-08 22:03:45,120 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-bottom 2023-04-08 22:03:45,129 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-bottom 2023-04-08 22:03:45,132 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c 2023-04-08 22:03:45,137 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3 2023-04-08 22:03:45,140 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa 2023-04-08 22:03:45,140 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(310): Store=cfac3b4c98d7f2cc5feb51a9996121dc/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:45,140 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,141 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2 2023-04-08 22:03:45,141 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2 2023-04-08 22:03:45,141 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region cfac3b4c98d7f2cc5feb51a9996121dc columnFamilyName cf2 2023-04-08 22:03:45,152 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-bottom 2023-04-08 22:03:45,156 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-bottom 2023-04-08 22:03:45,160 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,161 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-bottom 2023-04-08 22:03:45,166 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-bottom 2023-04-08 22:03:45,170 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9 2023-04-08 22:03:45,173 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640 2023-04-08 22:03:45,176 DEBUG [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe 2023-04-08 22:03:45,176 INFO [StoreOpener-cfac3b4c98d7f2cc5feb51a9996121dc-1] regionserver.HStore(310): Store=cfac3b4c98d7f2cc5feb51a9996121dc/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:45,177 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,178 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,179 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:45,180 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened cfac3b4c98d7f2cc5feb51a9996121dc; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68111498, jitterRate=0.014940410852432251}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:45,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:45,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc., pid=77, masterSystemTime=1680991425076 2023-04-08 22:03:45,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,183 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:45,183 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:45,183 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,183 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): cfac3b4c98d7f2cc5feb51a9996121dc/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:45,183 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): cfac3b4c98d7f2cc5feb51a9996121dc/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cfac3b4c98d7f2cc5feb51a9996121dc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of cfac3b4c98d7f2cc5feb51a9996121dc/cf2 in testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp, totalSize=35.0 K 2023-04-08 22:03:45,183 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp, totalSize=35.0 K 2023-04-08 22:03:45,184 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991422617 2023-04-08 22:03:45,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,184 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991422617 2023-04-08 22:03:45,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,184 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ae1a4e6d25e5fb5dd5133abebab5ecb7, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.', STARTKEY => '60', ENDKEY => ''} 2023-04-08 22:03:45,184 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=cfac3b4c98d7f2cc5feb51a9996121dc, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:45,185 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991425184"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991425184"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991425184"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991425184"}]},"ts":"1680991425184"} 2023-04-08 22:03:45,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:45,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:45,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,185 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,186 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991422802 2023-04-08 22:03:45,186 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991422802 2023-04-08 22:03:45,186 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991422992 2023-04-08 22:03:45,186 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991422992 2023-04-08 22:03:45,186 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991423164 2023-04-08 22:03:45,187 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991423164 2023-04-08 22:03:45,187 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,187 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991423347 2023-04-08 22:03:45,187 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991423347 2023-04-08 22:03:45,187 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991423545 2023-04-08 22:03:45,187 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991423545 2023-04-08 22:03:45,187 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1 2023-04-08 22:03:45,187 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1 2023-04-08 22:03:45,188 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991423744 2023-04-08 22:03:45,188 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991423744 2023-04-08 22:03:45,188 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae1a4e6d25e5fb5dd5133abebab5ecb7 columnFamilyName cf1 2023-04-08 22:03:45,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-04-08 22:03:45,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure cfac3b4c98d7f2cc5feb51a9996121dc, server=jenkins-hbase20.apache.org,41779,1680991398002 in 262 msec 2023-04-08 22:03:45,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, ASSIGN in 420 msec 2023-04-08 22:03:45,207 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): cfac3b4c98d7f2cc5feb51a9996121dc#cf1#compaction#72 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,207 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-top 2023-04-08 22:03:45,209 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): cfac3b4c98d7f2cc5feb51a9996121dc#cf2#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,212 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,212 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-top 2023-04-08 22:03:45,216 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-top 2023-04-08 22:03:45,218 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-top 2023-04-08 22:03:45,221 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:45,222 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:45,225 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:45,225 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:45,228 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:45,228 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:45,231 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(310): Store=ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:45,232 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,233 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2 2023-04-08 22:03:45,233 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2 2023-04-08 22:03:45,234 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ae1a4e6d25e5fb5dd5133abebab5ecb7 columnFamilyName cf2 2023-04-08 22:03:45,236 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:03:45,258 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp/cf1/045b9161339f4400971bc5930a81be39 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/045b9161339f4400971bc5930a81be39 2023-04-08 22:03:45,261 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp/cf2/585f9a44a7ed4c629af40860311d3c4b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/585f9a44a7ed4c629af40860311d3c4b 2023-04-08 22:03:45,263 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-top 2023-04-08 22:03:45,264 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cfac3b4c98d7f2cc5feb51a9996121dc/cf1 of cfac3b4c98d7f2cc5feb51a9996121dc into 045b9161339f4400971bc5930a81be39(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,264 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:45,264 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc., storeName=cfac3b4c98d7f2cc5feb51a9996121dc/cf1, priority=9, startTime=1680991425182; duration=0sec 2023-04-08 22:03:45,264 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,267 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in cfac3b4c98d7f2cc5feb51a9996121dc/cf2 of cfac3b4c98d7f2cc5feb51a9996121dc into 585f9a44a7ed4c629af40860311d3c4b(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,268 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-top 2023-04-08 22:03:45,268 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:45,268 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc., storeName=cfac3b4c98d7f2cc5feb51a9996121dc/cf2, priority=9, startTime=1680991425182; duration=0sec 2023-04-08 22:03:45,268 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,271 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,271 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-top 2023-04-08 22:03:45,275 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-top 2023-04-08 22:03:45,282 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:45,282 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:45,286 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:45,286 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:45,290 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:45,290 DEBUG [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:45,290 INFO [StoreOpener-ae1a4e6d25e5fb5dd5133abebab5ecb7-1] regionserver.HStore(310): Store=ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:45,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,294 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:03:45,295 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:45,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened ae1a4e6d25e5fb5dd5133abebab5ecb7; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66244320, jitterRate=-0.012882709503173828}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:03:45,296 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for ae1a4e6d25e5fb5dd5133abebab5ecb7: 2023-04-08 22:03:45,297 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7., pid=78, masterSystemTime=1680991425076 2023-04-08 22:03:45,297 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,298 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:45,298 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:03:45,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,298 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1 is initiating minor compaction (all files) 2023-04-08 22:03:45,298 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2 is initiating minor compaction (all files) 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/.tmp, totalSize=34.8 K 2023-04-08 22:03:45,298 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/.tmp, totalSize=34.8 K 2023-04-08 22:03:45,299 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991422617 2023-04-08 22:03:45,299 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991422617 2023-04-08 22:03:45,299 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991422802 2023-04-08 22:03:45,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,300 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991422802 2023-04-08 22:03:45,300 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991422992 2023-04-08 22:03:45,300 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=ae1a4e6d25e5fb5dd5133abebab5ecb7, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:45,300 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991422992 2023-04-08 22:03:45,300 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1680991423744 2023-04-08 22:03:45,300 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991425300"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991425300"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991425300"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991425300"}]},"ts":"1680991425300"} 2023-04-08 22:03:45,301 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1680991423744 2023-04-08 22:03:45,301 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,301 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,301 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,301 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,302 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,302 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-04-08 22:03:45,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-04-08 22:03:45,304 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure ae1a4e6d25e5fb5dd5133abebab5ecb7, server=jenkins-hbase20.apache.org,41779,1680991398002 in 374 msec 2023-04-08 22:03:45,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-04-08 22:03:45,306 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, ASSIGN in 535 msec 2023-04-08 22:03:45,307 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 in 1.0000 sec 2023-04-08 22:03:45,319 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): ae1a4e6d25e5fb5dd5133abebab5ecb7#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,319 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): ae1a4e6d25e5fb5dd5133abebab5ecb7#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,321 DEBUG [Listener at localhost.localdomain/46567] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-04-08 22:03:45,330 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 148.251.75.209:32998, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-04-08 22:03:45,330 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,331 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-08 22:03:45,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HStore(1912): cfac3b4c98d7f2cc5feb51a9996121dc/cf1 is initiating major compaction (all files) 2023-04-08 22:03:45,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@48e18d1d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:45,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-04-08 22:03:45,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HStore(1912): cfac3b4c98d7f2cc5feb51a9996121dc/cf2 is initiating major compaction (all files) 2023-04-08 22:03:45,332 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@39434fc0; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-08 22:03:45,334 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:45,334 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. because compaction request was cancelled 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-04-08 22:03:45,335 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. because compaction request was cancelled 2023-04-08 22:03:45,336 INFO [Listener at localhost.localdomain/46567] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-04-08 22:03:45,349 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd2a4323f544441eba92f979e9f344f6: NONE, but ROW specified in column family configuration 2023-04-08 22:03:45,350 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/.tmp/cf2/fd2a4323f544441eba92f979e9f344f6 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/fd2a4323f544441eba92f979e9f344f6 2023-04-08 22:03:45,354 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd2a4323f544441eba92f979e9f344f6: NONE, but ROW specified in column family configuration 2023-04-08 22:03:45,355 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2 of ae1a4e6d25e5fb5dd5133abebab5ecb7 into fd2a4323f544441eba92f979e9f344f6(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,355 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for ae1a4e6d25e5fb5dd5133abebab5ecb7: 2023-04-08 22:03:45,355 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7., storeName=ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2, priority=9, startTime=1680991425298; duration=0sec 2023-04-08 22:03:45,355 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-04-08 22:03:45,355 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of cfac3b4c98d7f2cc5feb51a9996121dc/cf1 in testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,356 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/045b9161339f4400971bc5930a81be39] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp, totalSize=6.8 K 2023-04-08 22:03:45,356 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 045b9161339f4400971bc5930a81be39, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991422617 2023-04-08 22:03:45,357 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. is not mergeable because it has references 2023-04-08 22:03:45,357 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. is not mergeable because it has references 2023-04-08 22:03:45,360 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 42459b1196274dba955102f3400e7e96: NONE, but ROW specified in column family configuration 2023-04-08 22:03:45,360 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): cfac3b4c98d7f2cc5feb51a9996121dc#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,360 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/.tmp/cf1/42459b1196274dba955102f3400e7e96 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/42459b1196274dba955102f3400e7e96 2023-04-08 22:03:45,364 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 42459b1196274dba955102f3400e7e96: NONE, but ROW specified in column family configuration 2023-04-08 22:03:45,365 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1 of ae1a4e6d25e5fb5dd5133abebab5ecb7 into 42459b1196274dba955102f3400e7e96(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,365 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for ae1a4e6d25e5fb5dd5133abebab5ecb7: 2023-04-08 22:03:45,365 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7., storeName=ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1, priority=9, startTime=1680991425297; duration=0sec 2023-04-08 22:03:45,365 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:03:45,365 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of cfac3b4c98d7f2cc5feb51a9996121dc/cf2 in testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,365 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/585f9a44a7ed4c629af40860311d3c4b] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp, totalSize=6.8 K 2023-04-08 22:03:45,366 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 585f9a44a7ed4c629af40860311d3c4b, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991422617 2023-04-08 22:03:45,370 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): cfac3b4c98d7f2cc5feb51a9996121dc#cf2#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:03:45,381 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp/cf2/853f2126fc09449b878b07c62f1a74a2 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/853f2126fc09449b878b07c62f1a74a2 2023-04-08 22:03:45,386 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in cfac3b4c98d7f2cc5feb51a9996121dc/cf2 of cfac3b4c98d7f2cc5feb51a9996121dc into 853f2126fc09449b878b07c62f1a74a2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,386 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:45,387 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1680991425332; duration=0sec 2023-04-08 22:03:45,387 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,468 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. is not mergeable because it has references 2023-04-08 22:03:45,469 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. is not mergeable because it has references 2023-04-08 22:03:45,582 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. is not mergeable because it has references 2023-04-08 22:03:45,583 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. is not mergeable because it has references 2023-04-08 22:03:45,693 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. is not mergeable because it has references 2023-04-08 22:03:45,694 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. is not mergeable because it has references 2023-04-08 22:03:45,786 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/.tmp/cf1/50ae94c19b474aa79c164c8bd88eec95 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/50ae94c19b474aa79c164c8bd88eec95 2023-04-08 22:03:45,793 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in cfac3b4c98d7f2cc5feb51a9996121dc/cf1 of cfac3b4c98d7f2cc5feb51a9996121dc into 50ae94c19b474aa79c164c8bd88eec95(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:03:45,793 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:45,794 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1680991425331; duration=0sec 2023-04-08 22:03:45,794 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:03:45,801 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. is not mergeable because it has references 2023-04-08 22:03:45,802 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=41779] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. is not mergeable because it has references 2023-04-08 22:03:45,807 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-04-08 22:03:45,809 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:45,810 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:45,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:45,811 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:45,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-08 22:03:45,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991425814"}]},"ts":"1680991425814"} 2023-04-08 22:03:45,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-04-08 22:03:45,815 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-04-08 22:03:45,816 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, UNASSIGN}] 2023-04-08 22:03:45,817 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, UNASSIGN 2023-04-08 22:03:45,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ae1a4e6d25e5fb5dd5133abebab5ecb7, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:45,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991425817"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991425817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991425817"}]},"ts":"1680991425817"} 2023-04-08 22:03:45,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure ae1a4e6d25e5fb5dd5133abebab5ecb7, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:45,820 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, UNASSIGN 2023-04-08 22:03:45,821 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=cfac3b4c98d7f2cc5feb51a9996121dc, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:03:45,821 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991425821"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991425821"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991425821"}]},"ts":"1680991425821"} 2023-04-08 22:03:45,822 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure cfac3b4c98d7f2cc5feb51a9996121dc, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:03:45,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-08 22:03:45,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing cfac3b4c98d7f2cc5feb51a9996121dc, disabling compactions & flushes 2023-04-08 22:03:45,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. after waiting 0 ms 2023-04-08 22:03:45,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:45,973 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/045b9161339f4400971bc5930a81be39, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-bottom] to archive 2023-04-08 22:03:45,975 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:45,980 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,982 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,983 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,985 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-f19a6beb8b0b46d8be9bc508884380fa 2023-04-08 22:03:45,987 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-c08744d7e7dc403c9666e129ac13cee3 2023-04-08 22:03:45,989 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-656c9bbb21614516a9ee1e82205cf07c 2023-04-08 22:03:45,990 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/045b9161339f4400971bc5930a81be39 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/045b9161339f4400971bc5930a81be39 2023-04-08 22:03:45,991 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,992 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/585f9a44a7ed4c629af40860311d3c4b, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-bottom] to archive 2023-04-08 22:03:45,993 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:45,995 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,996 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,997 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:45,998 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-207585c27ccc4a0b8b27a60ba79c3640 2023-04-08 22:03:45,999 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1f1871ccd13f4ac281e1a84c49580da9 2023-04-08 22:03:46,000 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-69b8350e1dd6467ba3fc208e2f408cbe 2023-04-08 22:03:46,001 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/585f9a44a7ed4c629af40860311d3c4b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/585f9a44a7ed4c629af40860311d3c4b 2023-04-08 22:03:46,001 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-04-08 22:03:46,005 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:46,006 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc. 2023-04-08 22:03:46,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for cfac3b4c98d7f2cc5feb51a9996121dc: 2023-04-08 22:03:46,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:46,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:46,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing ae1a4e6d25e5fb5dd5133abebab5ecb7, disabling compactions & flushes 2023-04-08 22:03:46,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:46,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:46,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. after waiting 0 ms 2023-04-08 22:03:46,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:46,008 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=cfac3b4c98d7f2cc5feb51a9996121dc, regionState=CLOSED 2023-04-08 22:03:46,008 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991426008"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991426008"}]},"ts":"1680991426008"} 2023-04-08 22:03:46,009 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605] to archive 2023-04-08 22:03:46,010 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:46,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-04-08 22:03:46,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure cfac3b4c98d7f2cc5feb51a9996121dc, server=jenkins-hbase20.apache.org,41779,1680991398002 in 187 msec 2023-04-08 22:03:46,011 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/ee19d9150cfd43768dcdb9d44c50df89.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,012 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=cfac3b4c98d7f2cc5feb51a9996121dc, UNASSIGN in 195 msec 2023-04-08 22:03:46,013 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/4e79ace01e9a42f5aac7f0ccb078f238.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,014 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/6a96fe0f504c475b88397a07c0c43291.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,015 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/63bf6ad8d3a94c5e89847f18667fefc0.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,016 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:46,017 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:46,018 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:46,020 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3] to archive 2023-04-08 22:03:46,020 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:03:46,022 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,023 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/7e2e1210cb8b45cdb30080e19244ed20.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,024 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/742ac42086de40749f2c35421eaab183.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,026 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/9dc1af50a9604610802cf1b540036654.ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,027 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:46,028 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:46,029 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/testSplitTableRegionDeletedRowsDaughter=ee9b8d200835a543c2b4c8142e56c58a-0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:46,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-04-08 22:03:46,034 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:46,035 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7. 2023-04-08 22:03:46,035 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for ae1a4e6d25e5fb5dd5133abebab5ecb7: 2023-04-08 22:03:46,037 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:46,037 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=ae1a4e6d25e5fb5dd5133abebab5ecb7, regionState=CLOSED 2023-04-08 22:03:46,038 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1680991426037"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991426037"}]},"ts":"1680991426037"} 2023-04-08 22:03:46,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-04-08 22:03:46,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure ae1a4e6d25e5fb5dd5133abebab5ecb7, server=jenkins-hbase20.apache.org,41779,1680991398002 in 221 msec 2023-04-08 22:03:46,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-04-08 22:03:46,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ae1a4e6d25e5fb5dd5133abebab5ecb7, UNASSIGN in 225 msec 2023-04-08 22:03:46,044 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991426044"}]},"ts":"1680991426044"} 2023-04-08 22:03:46,046 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-04-08 22:03:46,047 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-04-08 22:03:46,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 238 msec 2023-04-08 22:03:46,116 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-04-08 22:03:46,116 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-04-08 22:03:46,117 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,120 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,121 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,121 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-04-08 22:03:46,125 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,125 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:46,125 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:46,127 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/recovered.edits] 2023-04-08 22:03:46,127 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/recovered.edits] 2023-04-08 22:03:46,127 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/recovered.edits] 2023-04-08 22:03:46,129 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/42459b1196274dba955102f3400e7e96 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf1/42459b1196274dba955102f3400e7e96 2023-04-08 22:03:46,129 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/0eae0cd2866a4f71a78e067c3b866524 2023-04-08 22:03:46,129 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/50ae94c19b474aa79c164c8bd88eec95 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf1/50ae94c19b474aa79c164c8bd88eec95 2023-04-08 22:03:46,130 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/1088cfa683174ff7a7f67d0ca4e26705 2023-04-08 22:03:46,131 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/fd2a4323f544441eba92f979e9f344f6 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/cf2/fd2a4323f544441eba92f979e9f344f6 2023-04-08 22:03:46,131 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/4e79ace01e9a42f5aac7f0ccb078f238 2023-04-08 22:03:46,131 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/853f2126fc09449b878b07c62f1a74a2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/cf2/853f2126fc09449b878b07c62f1a74a2 2023-04-08 22:03:46,132 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/63bf6ad8d3a94c5e89847f18667fefc0 2023-04-08 22:03:46,133 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/656c9bbb21614516a9ee1e82205cf07c 2023-04-08 22:03:46,133 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/recovered.edits/111.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7/recovered.edits/111.seqid 2023-04-08 22:03:46,134 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/recovered.edits/113.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc/recovered.edits/113.seqid 2023-04-08 22:03:46,134 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:03:46,134 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/cfac3b4c98d7f2cc5feb51a9996121dc 2023-04-08 22:03:46,135 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/6a96fe0f504c475b88397a07c0c43291 2023-04-08 22:03:46,135 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/c08744d7e7dc403c9666e129ac13cee3 2023-04-08 22:03:46,136 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/cc40963b18f24df6b366ab36b62fc605 2023-04-08 22:03:46,137 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/ee19d9150cfd43768dcdb9d44c50df89 2023-04-08 22:03:46,138 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf1/f19a6beb8b0b46d8be9bc508884380fa 2023-04-08 22:03:46,139 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/0db9de72723849b4b02b8b38643f65c3 2023-04-08 22:03:46,140 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/1f1871ccd13f4ac281e1a84c49580da9 2023-04-08 22:03:46,141 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/207585c27ccc4a0b8b27a60ba79c3640 2023-04-08 22:03:46,142 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/69b8350e1dd6467ba3fc208e2f408cbe 2023-04-08 22:03:46,143 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/742ac42086de40749f2c35421eaab183 2023-04-08 22:03:46,144 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/762a0ea242d34ba1a5d7e74c2f2dbf09 2023-04-08 22:03:46,145 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/7e2e1210cb8b45cdb30080e19244ed20 2023-04-08 22:03:46,146 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/9dc1af50a9604610802cf1b540036654 2023-04-08 22:03:46,147 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/b32cbf9dfdf040ab9a1006917be8c630 2023-04-08 22:03:46,148 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/cf2/eda2be3f5b5844dbb9c6e3bbefaaf9cb 2023-04-08 22:03:46,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/recovered.edits/106.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a/recovered.edits/106.seqid 2023-04-08 22:03:46,150 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ee9b8d200835a543c2b4c8142e56c58a 2023-04-08 22:03:46,150 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-04-08 22:03:46,152 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,154 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-04-08 22:03:46,155 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-04-08 22:03:46,156 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,157 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-04-08 22:03:46,157 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991426157"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:46,157 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991426157"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:46,157 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991426157"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:46,159 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:03:46,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ee9b8d200835a543c2b4c8142e56c58a, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1680991422109.ee9b8d200835a543c2b4c8142e56c58a.', STARTKEY => '', ENDKEY => ''}, {ENCODED => cfac3b4c98d7f2cc5feb51a9996121dc, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1680991424306.cfac3b4c98d7f2cc5feb51a9996121dc.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => ae1a4e6d25e5fb5dd5133abebab5ecb7, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1680991424306.ae1a4e6d25e5fb5dd5133abebab5ecb7.', STARTKEY => '60', ENDKEY => ''}] 2023-04-08 22:03:46,159 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-04-08 22:03:46,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991426159"}]},"ts":"9223372036854775807"} 2023-04-08 22:03:46,160 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-04-08 22:03:46,161 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:03:46,162 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 45 msec 2023-04-08 22:03:46,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-04-08 22:03:46,222 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-04-08 22:03:46,240 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 404) Potentially hanging thread: RS:2;jenkins-hbase20:41779-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:41490 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:37410 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:51392 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:37414 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:58650 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:37486 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:51338 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:51320 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:51330 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:58640 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x49d8264d-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:58430 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:41474 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:58390 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:37396 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:51366 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_1304663717_15 at /127.0.0.1:37442 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=863 (was 855) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 250), ProcessCount=163 (was 165), AvailableMemoryMB=4869 (was 4169) - AvailableMemoryMB LEAK? - 2023-04-08 22:03:46,256 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=863, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=163, AvailableMemoryMB=4869 2023-04-08 22:03:46,257 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:03:46,259 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:03:46,260 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:03:46,261 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:03:46,263 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:03:46,264 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:03:46,265 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,266 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f empty. 2023-04-08 22:03:46,267 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,267 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-04-08 22:03:46,279 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:46,280 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 2e5e8a708bb5782faafc9eb08d64e11f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:03:46,289 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:46,289 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 2e5e8a708bb5782faafc9eb08d64e11f, disabling compactions & flushes 2023-04-08 22:03:46,289 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,289 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,290 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. after waiting 0 ms 2023-04-08 22:03:46,290 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,290 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,290 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 2e5e8a708bb5782faafc9eb08d64e11f: 2023-04-08 22:03:46,292 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:03:46,293 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991426293"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991426293"}]},"ts":"1680991426293"} 2023-04-08 22:03:46,295 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:03:46,295 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:03:46,296 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991426296"}]},"ts":"1680991426296"} 2023-04-08 22:03:46,296 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-04-08 22:03:46,300 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:46,300 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:46,300 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:46,301 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:46,301 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:03:46,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, ASSIGN}] 2023-04-08 22:03:46,302 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, ASSIGN 2023-04-08 22:03:46,303 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:03:46,453 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:46,458 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:46,458 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991426458"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991426458"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991426458"}]},"ts":"1680991426458"} 2023-04-08 22:03:46,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:46,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2e5e8a708bb5782faafc9eb08d64e11f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:46,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:46,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:46,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,622 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,626 DEBUG [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:03:46,626 DEBUG [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:03:46,626 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e5e8a708bb5782faafc9eb08d64e11f columnFamilyName cf1 2023-04-08 22:03:46,628 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] regionserver.HStore(310): Store=2e5e8a708bb5782faafc9eb08d64e11f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:46,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:03:46,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 2e5e8a708bb5782faafc9eb08d64e11f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60692939, jitterRate=-0.09560473263263702}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-08 22:03:46,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 2e5e8a708bb5782faafc9eb08d64e11f: 2023-04-08 22:03:46,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f., pid=87, masterSystemTime=1680991426612 2023-04-08 22:03:46,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:46,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991426638"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991426638"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991426638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991426638"}]},"ts":"1680991426638"} 2023-04-08 22:03:46,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-04-08 22:03:46,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967 in 180 msec 2023-04-08 22:03:46,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-04-08 22:03:46,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, ASSIGN in 341 msec 2023-04-08 22:03:46,645 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:03:46,645 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991426645"}]},"ts":"1680991426645"} 2023-04-08 22:03:46,647 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-04-08 22:03:46,649 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:03:46,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 389 msec 2023-04-08 22:03:46,778 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$12(2454): Client=jenkins//148.251.75.209 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-04-08 22:03:46,788 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:03:46,799 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-04-08 22:03:46,800 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-04-08 22:03:46,800 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-04-08 22:03:46,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-04-08 22:03:46,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, REOPEN/MOVE}] 2023-04-08 22:03:46,816 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, REOPEN/MOVE 2023-04-08 22:03:46,817 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:46,817 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991426817"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991426817"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991426817"}]},"ts":"1680991426817"} 2023-04-08 22:03:46,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:46,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 2e5e8a708bb5782faafc9eb08d64e11f, disabling compactions & flushes 2023-04-08 22:03:46,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,972 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. after waiting 0 ms 2023-04-08 22:03:46,973 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-04-08 22:03:46,981 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:03:46,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:46,982 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 2e5e8a708bb5782faafc9eb08d64e11f: 2023-04-08 22:03:46,982 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 2e5e8a708bb5782faafc9eb08d64e11f to self. 2023-04-08 22:03:46,985 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:46,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=CLOSED 2023-04-08 22:03:46,986 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991426985"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991426985"}]},"ts":"1680991426985"} 2023-04-08 22:03:46,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-04-08 22:03:46,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967 in 169 msec 2023-04-08 22:03:46,990 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=true 2023-04-08 22:03:47,140 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:03:47,141 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:47,141 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991427141"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991427141"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991427141"}]},"ts":"1680991427141"} 2023-04-08 22:03:47,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:03:47,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:47,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2e5e8a708bb5782faafc9eb08d64e11f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:03:47,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:47,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,309 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,312 DEBUG [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:03:47,312 DEBUG [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:03:47,313 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e5e8a708bb5782faafc9eb08d64e11f columnFamilyName cf1 2023-04-08 22:03:47,314 INFO [StoreOpener-2e5e8a708bb5782faafc9eb08d64e11f-1] regionserver.HStore(310): Store=2e5e8a708bb5782faafc9eb08d64e11f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:03:47,315 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,316 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,320 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:03:47,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 2e5e8a708bb5782faafc9eb08d64e11f; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73280838, jitterRate=0.0919695794582367}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-08 22:03:47,322 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 2e5e8a708bb5782faafc9eb08d64e11f: 2023-04-08 22:03:47,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f., pid=92, masterSystemTime=1680991427299 2023-04-08 22:03:47,324 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:47,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:03:47,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:03:47,325 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991427325"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991427325"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991427325"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991427325"}]},"ts":"1680991427325"} 2023-04-08 22:03:47,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-04-08 22:03:47,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967 in 182 msec 2023-04-08 22:03:47,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-04-08 22:03:47,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, REOPEN/MOVE in 515 msec 2023-04-08 22:03:47,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-04-08 22:03:47,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 532 msec 2023-04-08 22:03:47,339 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, ASSIGN}] 2023-04-08 22:03:47,340 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, ASSIGN 2023-04-08 22:03:47,341 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-04-08 22:03:47,491 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:03:47,494 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:03:47,494 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:03:47,494 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:03:47,494 DEBUG [jenkins-hbase20:39273] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-04-08 22:03:47,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=33a379c734056336fa36cffbc96b48b1, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:03:47,497 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991427497"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1680991427497"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1680991427497"}]},"ts":"1680991427497"} 2023-04-08 22:03:47,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 33a379c734056336fa36cffbc96b48b1, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:03:47,661 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:03:47,661 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 33a379c734056336fa36cffbc96b48b1, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-04-08 22:03:47,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:03:47,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:03:47,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:03:47,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:03:47,663 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:03:47,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:47,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:47,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:47,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:48,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,316 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-04-08 22:03:49,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:49,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,272 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,590 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:03:50,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:50,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:51,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,178 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:52,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:53,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:54,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:55,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:56,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:57,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:58,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:03:59,902 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,002 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,403 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:00,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:01,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:02,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,112 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,613 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:03,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:04,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,118 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:05,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:06,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:07,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:08,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:09,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:10,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:11,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:12,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:13,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:14,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:15,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,356 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:16,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,559 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:17,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,563 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,663 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,763 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:18,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,064 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,164 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,312 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-08 22:04:19,321 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-08 22:04:19,337 DEBUG [regionserver/jenkins-hbase20:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-04-08 22:04:19,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:19,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:20,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,673 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,773 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,174 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,274 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:22,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:23,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:24,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:25,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:26,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:27,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,293 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,794 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:28,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,898 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:29,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,199 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,299 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,399 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:30,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,804 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:31,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:32,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:33,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:34,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,114 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,615 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:35,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,619 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:36,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:37,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:38,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:39,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:40,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:41,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:42,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:43,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:44,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,446 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:45,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,450 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,851 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,896 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-08 22:04:46,951 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:46,998 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-08 22:04:47,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,201 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-08 22:04:47,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,352 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,504 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-08 22:04:47,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:47,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-04-08 22:04:48,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,456 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,857 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:48,957 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:49,008 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:49,009 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:49,009 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:49,010 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:49,010 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:49,011 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 2e5e8a708bb5782faafc9eb08d64e11f, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:49,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 exec-time=5 msec 2023-04-08 22:04:49,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:49,158 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:49,258 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-04-08 22:04:49,359 INFO [StoreOpener-33a379c734056336fa36cffbc96b48b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:04:49,367 DEBUG [StoreOpener-33a379c734056336fa36cffbc96b48b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:04:49,367 DEBUG [StoreOpener-33a379c734056336fa36cffbc96b48b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1 2023-04-08 22:04:49,368 INFO [StoreOpener-33a379c734056336fa36cffbc96b48b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 33a379c734056336fa36cffbc96b48b1 columnFamilyName cf1 2023-04-08 22:04:49,371 INFO [StoreOpener-33a379c734056336fa36cffbc96b48b1-1] regionserver.HStore(310): Store=33a379c734056336fa36cffbc96b48b1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:49,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 33a379c734056336fa36cffbc96b48b1; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74969120, jitterRate=0.1171269416809082}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-04-08 22:04:49,374 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 33a379c734056336fa36cffbc96b48b1: 2023-04-08 22:04:49,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1., pid=94, masterSystemTime=1680991427654 2023-04-08 22:04:49,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:49,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:49,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=33a379c734056336fa36cffbc96b48b1, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:49,376 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991489375"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1680991489375"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1680991489375"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1680991489375"}]},"ts":"1680991489375"} 2023-04-08 22:04:49,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-04-08 22:04:49,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 33a379c734056336fa36cffbc96b48b1, server=jenkins-hbase20.apache.org,43551,1680991397921 in 1 mins, 1.878 sec 2023-04-08 22:04:49,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-04-08 22:04:49,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, ASSIGN in 1 mins, 2.039 sec 2023-04-08 22:04:49,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.598 sec 2023-04-08 22:04:49,381 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1680991426774","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62607,"client":"148.251.75.209:50868","queuetimems":0,"class":"HMaster"} 2023-04-08 22:04:49,381 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 633, skipping over this response of 3 bytes 2023-04-08 22:04:51,260 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up 2023-04-08 22:04:51,263 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:04:51,268 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,269 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,271 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-08 22:04:51,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991491274"}]},"ts":"1680991491274"} 2023-04-08 22:04:51,276 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-04-08 22:04:51,278 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-04-08 22:04:51,279 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, UNASSIGN}] 2023-04-08 22:04:51,280 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, UNASSIGN 2023-04-08 22:04:51,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=33a379c734056336fa36cffbc96b48b1, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:51,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991491280"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1680991491280"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1680991491280"}]},"ts":"1680991491280"} 2023-04-08 22:04:51,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 33a379c734056336fa36cffbc96b48b1, server=jenkins-hbase20.apache.org,43551,1680991397921}] 2023-04-08 22:04:51,286 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, UNASSIGN 2023-04-08 22:04:51,287 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:51,287 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991491287"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991491287"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991491287"}]},"ts":"1680991491287"} 2023-04-08 22:04:51,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:04:51,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-08 22:04:51,436 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:04:51,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 33a379c734056336fa36cffbc96b48b1, disabling compactions & flushes 2023-04-08 22:04:51,438 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:51,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:51,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. after waiting 0 ms 2023-04-08 22:04:51,438 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:51,439 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:51,440 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1. 2023-04-08 22:04:51,441 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 33a379c734056336fa36cffbc96b48b1: 2023-04-08 22:04:51,442 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:04:51,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 2e5e8a708bb5782faafc9eb08d64e11f, disabling compactions & flushes 2023-04-08 22:04:51,443 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:04:51,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:04:51,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. after waiting 0 ms 2023-04-08 22:04:51,443 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:04:51,444 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 33a379c734056336fa36cffbc96b48b1 2023-04-08 22:04:51,445 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=33a379c734056336fa36cffbc96b48b1, regionState=CLOSED 2023-04-08 22:04:51,446 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991491445"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1680991491445"}]},"ts":"1680991491445"} 2023-04-08 22:04:51,451 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-04-08 22:04:51,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:51,453 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f. 2023-04-08 22:04:51,453 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 2e5e8a708bb5782faafc9eb08d64e11f: 2023-04-08 22:04:51,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-04-08 22:04:51,453 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 33a379c734056336fa36cffbc96b48b1, server=jenkins-hbase20.apache.org,43551,1680991397921 in 168 msec 2023-04-08 22:04:51,454 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:04:51,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1, UNASSIGN in 174 msec 2023-04-08 22:04:51,455 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=2e5e8a708bb5782faafc9eb08d64e11f, regionState=CLOSED 2023-04-08 22:04:51,455 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1680991491455"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991491455"}]},"ts":"1680991491455"} 2023-04-08 22:04:51,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-04-08 22:04:51,458 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 2e5e8a708bb5782faafc9eb08d64e11f, server=jenkins-hbase20.apache.org,41749,1680991397967 in 168 msec 2023-04-08 22:04:51,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-04-08 22:04:51,460 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e5e8a708bb5782faafc9eb08d64e11f, UNASSIGN in 179 msec 2023-04-08 22:04:51,461 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991491461"}]},"ts":"1680991491461"} 2023-04-08 22:04:51,462 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-04-08 22:04:51,463 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-04-08 22:04:51,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 195 msec 2023-04-08 22:04:51,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-04-08 22:04:51,578 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-04-08 22:04:51,580 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,581 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,584 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,585 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-04-08 22:04:51,594 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:04:51,597 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits] 2023-04-08 22:04:51,606 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits/7.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f/recovered.edits/7.seqid 2023-04-08 22:04:51,607 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/2e5e8a708bb5782faafc9eb08d64e11f 2023-04-08 22:04:51,607 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-04-08 22:04:51,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,612 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-04-08 22:04:51,614 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-04-08 22:04:51,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,615 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-04-08 22:04:51,615 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991491615"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:51,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991491615"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:51,617 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-04-08 22:04:51,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 2e5e8a708bb5782faafc9eb08d64e11f, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260.2e5e8a708bb5782faafc9eb08d64e11f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 33a379c734056336fa36cffbc96b48b1, NAME => 'testRollbackForSplitTableRegionWithReplica,,1680991426260_0001.33a379c734056336fa36cffbc96b48b1.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-04-08 22:04:51,617 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-04-08 22:04:51,618 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991491617"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:51,619 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-04-08 22:04:51,620 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:51,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 40 msec 2023-04-08 22:04:51,688 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-04-08 22:04:51,689 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-04-08 22:04:51,707 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=346 (was 414), OpenFileDescriptor=656 (was 863), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 229), ProcessCount=163 (was 163), AvailableMemoryMB=4874 (was 4869) - AvailableMemoryMB LEAK? - 2023-04-08 22:04:51,720 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=347, OpenFileDescriptor=656, MaxFileDescriptor=60000, SystemLoadAverage=86, ProcessCount=163, AvailableMemoryMB=4873 2023-04-08 22:04:51,721 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:04:51,723 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:04:51,724 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:04:51,725 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:51,726 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:04:51,727 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:04:51,728 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:51,729 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c empty. 2023-04-08 22:04:51,729 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:51,729 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-04-08 22:04:51,740 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-04-08 22:04:51,741 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => d139e42a2228cc575ed1b3eefa4f419c, NAME => 'testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing d139e42a2228cc575ed1b3eefa4f419c, disabling compactions & flushes 2023-04-08 22:04:51,750 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. after waiting 0 ms 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:51,750 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:51,750 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:51,752 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:04:51,753 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1680991491753"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991491753"}]},"ts":"1680991491753"} 2023-04-08 22:04:51,754 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:04:51,755 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:04:51,755 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991491755"}]},"ts":"1680991491755"} 2023-04-08 22:04:51,756 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-04-08 22:04:51,757 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:04:51,758 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:04:51,758 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:04:51,758 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:04:51,758 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:04:51,758 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, ASSIGN}] 2023-04-08 22:04:51,760 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, ASSIGN 2023-04-08 22:04:51,760 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:04:51,910 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:04:51,911 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=d139e42a2228cc575ed1b3eefa4f419c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:51,912 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1680991491911"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991491911"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991491911"}]},"ts":"1680991491911"} 2023-04-08 22:04:51,913 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure d139e42a2228cc575ed1b3eefa4f419c, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:52,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d139e42a2228cc575ed1b3eefa4f419c, NAME => 'testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:04:52,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:04:52,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,078 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:52,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,079 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,080 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,082 DEBUG [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1 2023-04-08 22:04:52,082 DEBUG [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1 2023-04-08 22:04:52,083 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d139e42a2228cc575ed1b3eefa4f419c columnFamilyName cf1 2023-04-08 22:04:52,083 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] regionserver.HStore(310): Store=d139e42a2228cc575ed1b3eefa4f419c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:52,083 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,085 DEBUG [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2 2023-04-08 22:04:52,085 DEBUG [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2 2023-04-08 22:04:52,085 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d139e42a2228cc575ed1b3eefa4f419c columnFamilyName cf2 2023-04-08 22:04:52,086 INFO [StoreOpener-d139e42a2228cc575ed1b3eefa4f419c-1] regionserver.HStore(310): Store=d139e42a2228cc575ed1b3eefa4f419c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:52,086 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,087 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,088 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:04:52,089 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:52,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:04:52,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened d139e42a2228cc575ed1b3eefa4f419c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73943347, jitterRate=0.10184173285961151}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:04:52,091 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:52,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c., pid=104, masterSystemTime=1680991492066 2023-04-08 22:04:52,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,093 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=d139e42a2228cc575ed1b3eefa4f419c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,093 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1680991492093"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991492093"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991492093"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991492093"}]},"ts":"1680991492093"} 2023-04-08 22:04:52,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-04-08 22:04:52,096 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure d139e42a2228cc575ed1b3eefa4f419c, server=jenkins-hbase20.apache.org,41779,1680991398002 in 181 msec 2023-04-08 22:04:52,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-04-08 22:04:52,098 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, ASSIGN in 338 msec 2023-04-08 22:04:52,098 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:04:52,098 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991492098"}]},"ts":"1680991492098"} 2023-04-08 22:04:52,099 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-04-08 22:04:52,101 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:04:52,102 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 377 msec 2023-04-08 22:04:52,240 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:52,241 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:52,241 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:52,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:52,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:52,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,241 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,242 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,242 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,242 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,243 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,244 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:52,244 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:52,244 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,244 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:52,245 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:04:52,245 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,245 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,245 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,246 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,246 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,246 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,253 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/4b57f4b49e2c40a3b62170c148fb033f 2023-04-08 22:04:52,264 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/9eca576ea8ba4799a5234dc573bb8ac9 2023-04-08 22:04:52,269 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/4b57f4b49e2c40a3b62170c148fb033f as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/4b57f4b49e2c40a3b62170c148fb033f 2023-04-08 22:04:52,274 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/4b57f4b49e2c40a3b62170c148fb033f, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:04:52,275 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/9eca576ea8ba4799a5234dc573bb8ac9 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/9eca576ea8ba4799a5234dc573bb8ac9 2023-04-08 22:04:52,280 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/9eca576ea8ba4799a5234dc573bb8ac9, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:04:52,281 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 36ms, sequenceid=6, compaction requested=false 2023-04-08 22:04:52,281 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-04-08 22:04:52,282 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:52,282 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,282 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,283 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,283 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,284 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,284 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,285 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:52,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@306dea21[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:52,286 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:52,286 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,286 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,287 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:52,287 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:52,288 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,288 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:52,290 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:52,290 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:52,290 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:52,290 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:52,290 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:52,291 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:52,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,292 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,292 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,294 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,294 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,294 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:52,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,296 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:52,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,297 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,397 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:52,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:52,419 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:52,420 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:52,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:52,421 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:52,421 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,421 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:52,421 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:52,421 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,421 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,423 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,424 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,424 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,423 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,425 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,425 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,426 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,426 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,426 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,427 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,426 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,427 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,427 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,428 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:52,428 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:52,428 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,428 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:52,428 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:52,428 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,429 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,429 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,429 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,430 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,430 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,439 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/5641bcf3e1874a62b20fe6d072ab7fda 2023-04-08 22:04:52,451 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/d5aeac2644fc4f06a4b1310da2eb0775 2023-04-08 22:04:52,455 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/5641bcf3e1874a62b20fe6d072ab7fda as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda 2023-04-08 22:04:52,460 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:04:52,461 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/d5aeac2644fc4f06a4b1310da2eb0775 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775 2023-04-08 22:04:52,468 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:04:52,469 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 41ms, sequenceid=19, compaction requested=false 2023-04-08 22:04:52,469 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:52,469 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:52,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:52,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,469 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,471 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,471 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,472 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:52,475 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d4572d6[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:52,475 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:52,475 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:52,478 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:52,478 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,479 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,480 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,480 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,481 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,481 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,481 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:52,488 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:52,488 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:52,488 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:52,488 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:52,488 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:52,488 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:52,497 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,497 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,497 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,497 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,497 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,497 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,498 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,499 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,499 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,499 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,500 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,500 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,504 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,504 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,505 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:52,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:52,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:52,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,510 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:52,610 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:52,612 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:52,626 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:52,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:52,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:52,628 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:52,628 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,628 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:52,628 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:52,628 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,628 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,629 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,629 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,630 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,631 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,631 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:52,631 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:52,631 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,631 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:52,632 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:52,632 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,632 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,632 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,633 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,633 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,633 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,642 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/1cea22882e4743498b2cb237c3714c59 2023-04-08 22:04:52,659 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/ed378c157e724cc598a9a784acea631a 2023-04-08 22:04:52,665 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/1cea22882e4743498b2cb237c3714c59 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/1cea22882e4743498b2cb237c3714c59 2023-04-08 22:04:52,672 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/1cea22882e4743498b2cb237c3714c59, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:04:52,673 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/ed378c157e724cc598a9a784acea631a as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/ed378c157e724cc598a9a784acea631a 2023-04-08 22:04:52,679 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/ed378c157e724cc598a9a784acea631a, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:04:52,680 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 48ms, sequenceid=32, compaction requested=true 2023-04-08 22:04:52,680 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:52,680 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:52,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:52,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,680 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,682 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,682 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,683 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,683 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:52,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64b9cfde[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:52,686 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:52,686 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,689 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,689 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:52,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,690 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,691 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,691 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:52,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:52,694 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:52,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:52,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:52,694 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:52,694 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,696 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,695 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,696 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,696 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:52,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:52,701 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:52,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,701 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,702 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,802 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:52,803 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:52,813 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:52,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:52,815 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:52,815 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:52,815 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,815 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:52,815 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:52,816 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,816 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,816 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,816 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,816 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,816 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,817 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,817 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:52,818 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:52,818 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,818 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:52,819 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,819 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,827 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/57d230f4b6e74894aa021228fdf175ad 2023-04-08 22:04:52,837 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/b46b6b67c6374d7fa957999e81091a57 2023-04-08 22:04:52,841 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/57d230f4b6e74894aa021228fdf175ad as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/57d230f4b6e74894aa021228fdf175ad 2023-04-08 22:04:52,845 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/57d230f4b6e74894aa021228fdf175ad, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:04:52,845 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/b46b6b67c6374d7fa957999e81091a57 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/b46b6b67c6374d7fa957999e81091a57 2023-04-08 22:04:52,850 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/b46b6b67c6374d7fa957999e81091a57, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:04:52,851 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 32ms, sequenceid=45, compaction requested=true 2023-04-08 22:04:52,851 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:52,851 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,851 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:52,851 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:52,851 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,851 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,851 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,852 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,852 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,852 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,852 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:52,854 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9d962f9[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:52,854 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:52,854 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,855 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,855 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:52,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:52,855 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,856 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,856 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:52,858 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:52,858 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:52,858 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:52,858 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:52,859 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:52,859 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:52,859 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,859 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:52,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:52,859 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,859 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,859 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,859 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,860 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,860 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:52,861 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:52,862 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:52,862 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,862 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,864 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,864 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:52,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:52,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:52,865 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:52,965 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:52,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:52,976 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:52,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:52,977 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:52,977 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:52,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:52,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:52,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,979 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,979 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,979 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:52,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:52,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:52,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,980 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:52,980 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,980 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,980 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,980 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,980 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:52,981 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:52,981 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:52,981 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:52,981 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,982 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,982 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,982 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,982 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:52,982 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:52,992 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/6e52fd5d7c0647f79c35c03cd49853f8 2023-04-08 22:04:53,002 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/667a723adb7348a499c89f69c1de58bf 2023-04-08 22:04:53,005 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/6e52fd5d7c0647f79c35c03cd49853f8 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/6e52fd5d7c0647f79c35c03cd49853f8 2023-04-08 22:04:53,008 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/6e52fd5d7c0647f79c35c03cd49853f8, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:04:53,009 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/667a723adb7348a499c89f69c1de58bf as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/667a723adb7348a499c89f69c1de58bf 2023-04-08 22:04:53,013 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/667a723adb7348a499c89f69c1de58bf, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:04:53,014 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 32ms, sequenceid=58, compaction requested=true 2023-04-08 22:04:53,014 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:53,014 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,014 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:53,014 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:53,014 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,014 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:53,014 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:53,016 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,016 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,016 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,016 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:53,018 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56babd01[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:53,018 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:53,018 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,019 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:53,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,019 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:53,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:53,019 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,020 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,020 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,021 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,023 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:53,023 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:53,023 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:53,023 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:53,023 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:53,023 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:53,023 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,024 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,025 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,025 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,026 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,027 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,027 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,027 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:53,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,029 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:53,029 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,030 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,130 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:53,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:53,144 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:53,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:53,146 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-04-08 22:04:53,146 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-04-08 22:04:53,146 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:53,146 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-04-08 22:04:53,146 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-04-08 22:04:53,147 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,147 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,148 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,148 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,148 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,148 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,148 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,149 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,149 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,149 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,149 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:53,150 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-04-08 22:04:53,150 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:53,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:53,151 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-04-08 22:04:53,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:53,151 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-04-08 22:04:53,151 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:53,152 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:53,151 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. started... 2023-04-08 22:04:53,152 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:53,152 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,153 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,153 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,153 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:53,153 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,153 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:53,161 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/d59882a6bf6f42e4911d69320fb9c550 2023-04-08 22:04:53,172 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/d42cd4ebd90c4d10ac788808785c0036 2023-04-08 22:04:53,177 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/d59882a6bf6f42e4911d69320fb9c550 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d59882a6bf6f42e4911d69320fb9c550 2023-04-08 22:04:53,182 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d59882a6bf6f42e4911d69320fb9c550, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:04:53,183 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/d42cd4ebd90c4d10ac788808785c0036 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d42cd4ebd90c4d10ac788808785c0036 2023-04-08 22:04:53,186 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d42cd4ebd90c4d10ac788808785c0036, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:04:53,187 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 35ms, sequenceid=71, compaction requested=true 2023-04-08 22:04:53,187 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:53,187 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,187 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:53,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:53,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-04-08 22:04:53,188 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-04-08 22:04:53,189 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,189 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,189 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,189 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:53,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-04-08 22:04:53,192 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@209fa6c2[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:53,192 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-04-08 22:04:53,192 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,192 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,193 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:53,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:53,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-04-08 22:04:53,193 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,194 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,194 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,195 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-04-08 22:04:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:53,198 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-04-08 22:04:53,198 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-04-08 22:04:53,198 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:53,198 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:53,198 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-04-08 22:04:53,198 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:53,201 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,200 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,201 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,202 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:53,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:53,204 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-04-08 22:04:53,204 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,205 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,211 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-04-08 22:04:53,212 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:53,211 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:53,212 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,212 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:53,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:53,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:53,213 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:53,212 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,213 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:53,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:53,213 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-04-08 22:04:53,214 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-04-08 22:04:53,313 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:53,315 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:53,330 INFO [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,331 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda because midkey is the same as first or last row 2023-04-08 22:04:53,331 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775 because midkey is the same as first or last row 2023-04-08 22:04:53,332 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(2745): Flushing d139e42a2228cc575ed1b3eefa4f419c 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:04:53,341 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/d05239c88e1f4eebb105cb664c30baaa 2023-04-08 22:04:53,355 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/553f0b9679374ab58e6a70cd0f69f768 2023-04-08 22:04:53,361 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf1/d05239c88e1f4eebb105cb664c30baaa as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d05239c88e1f4eebb105cb664c30baaa 2023-04-08 22:04:53,365 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d05239c88e1f4eebb105cb664c30baaa, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:04:53,366 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/.tmp/cf2/553f0b9679374ab58e6a70cd0f69f768 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/553f0b9679374ab58e6a70cd0f69f768 2023-04-08 22:04:53,370 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/553f0b9679374ab58e6a70cd0f69f768, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:04:53,371 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d139e42a2228cc575ed1b3eefa4f419c in 40ms, sequenceid=82, compaction requested=true 2023-04-08 22:04:53,372 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.HRegion(2446): Flush status journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:53,372 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda because midkey is the same as first or last row 2023-04-08 22:04:53,372 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=41779] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775 because midkey is the same as first or last row 2023-04-08 22:04:53,372 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,372 DEBUG [Listener at localhost.localdomain/46567] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-04-08 22:04:53,372 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:04:53,374 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-04-08 22:04:53,374 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testInvalidSplitKey 2023-04-08 22:04:53,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-08 22:04:53,377 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991493377"}]},"ts":"1680991493377"} 2023-04-08 22:04:53,378 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-04-08 22:04:53,379 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-04-08 22:04:53,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, UNASSIGN}] 2023-04-08 22:04:53,380 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, UNASSIGN 2023-04-08 22:04:53,381 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d139e42a2228cc575ed1b3eefa4f419c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:53,381 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1680991493380"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991493380"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991493380"}]},"ts":"1680991493380"} 2023-04-08 22:04:53,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure d139e42a2228cc575ed1b3eefa4f419c, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:53,479 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-08 22:04:53,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:53,536 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing d139e42a2228cc575ed1b3eefa4f419c, disabling compactions & flushes 2023-04-08 22:04:53,536 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. after waiting 0 ms 2023-04-08 22:04:53,537 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,547 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:04:53,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:53,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c. 2023-04-08 22:04:53,548 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for d139e42a2228cc575ed1b3eefa4f419c: 2023-04-08 22:04:53,549 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:53,550 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=d139e42a2228cc575ed1b3eefa4f419c, regionState=CLOSED 2023-04-08 22:04:53,550 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1680991493550"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991493550"}]},"ts":"1680991493550"} 2023-04-08 22:04:53,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-04-08 22:04:53,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure d139e42a2228cc575ed1b3eefa4f419c, server=jenkins-hbase20.apache.org,41779,1680991398002 in 169 msec 2023-04-08 22:04:53,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-04-08 22:04:53,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=d139e42a2228cc575ed1b3eefa4f419c, UNASSIGN in 173 msec 2023-04-08 22:04:53,556 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991493556"}]},"ts":"1680991493556"} 2023-04-08 22:04:53,557 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-04-08 22:04:53,558 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-04-08 22:04:53,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 185 msec 2023-04-08 22:04:53,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-04-08 22:04:53,681 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-04-08 22:04:53,681 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testInvalidSplitKey 2023-04-08 22:04:53,682 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,683 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,684 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,684 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-04-08 22:04:53,688 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:53,690 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/recovered.edits] 2023-04-08 22:04:53,695 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/1cea22882e4743498b2cb237c3714c59 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/1cea22882e4743498b2cb237c3714c59 2023-04-08 22:04:53,697 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/4b57f4b49e2c40a3b62170c148fb033f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/4b57f4b49e2c40a3b62170c148fb033f 2023-04-08 22:04:53,698 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/5641bcf3e1874a62b20fe6d072ab7fda 2023-04-08 22:04:53,700 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/57d230f4b6e74894aa021228fdf175ad to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/57d230f4b6e74894aa021228fdf175ad 2023-04-08 22:04:53,702 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/6e52fd5d7c0647f79c35c03cd49853f8 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/6e52fd5d7c0647f79c35c03cd49853f8 2023-04-08 22:04:53,704 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d05239c88e1f4eebb105cb664c30baaa to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d05239c88e1f4eebb105cb664c30baaa 2023-04-08 22:04:53,706 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d59882a6bf6f42e4911d69320fb9c550 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf1/d59882a6bf6f42e4911d69320fb9c550 2023-04-08 22:04:53,709 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/553f0b9679374ab58e6a70cd0f69f768 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/553f0b9679374ab58e6a70cd0f69f768 2023-04-08 22:04:53,710 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/667a723adb7348a499c89f69c1de58bf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/667a723adb7348a499c89f69c1de58bf 2023-04-08 22:04:53,711 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/9eca576ea8ba4799a5234dc573bb8ac9 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/9eca576ea8ba4799a5234dc573bb8ac9 2023-04-08 22:04:53,712 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/b46b6b67c6374d7fa957999e81091a57 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/b46b6b67c6374d7fa957999e81091a57 2023-04-08 22:04:53,713 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d42cd4ebd90c4d10ac788808785c0036 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d42cd4ebd90c4d10ac788808785c0036 2023-04-08 22:04:53,715 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/d5aeac2644fc4f06a4b1310da2eb0775 2023-04-08 22:04:53,716 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/ed378c157e724cc598a9a784acea631a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/cf2/ed378c157e724cc598a9a784acea631a 2023-04-08 22:04:53,719 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c/recovered.edits/85.seqid 2023-04-08 22:04:53,719 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testInvalidSplitKey/d139e42a2228cc575ed1b3eefa4f419c 2023-04-08 22:04:53,720 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-04-08 22:04:53,722 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,724 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-04-08 22:04:53,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-04-08 22:04:53,729 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,729 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-04-08 22:04:53,729 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991493729"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:53,731 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-04-08 22:04:53,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d139e42a2228cc575ed1b3eefa4f419c, NAME => 'testInvalidSplitKey,,1680991491724.d139e42a2228cc575ed1b3eefa4f419c.', STARTKEY => '', ENDKEY => ''}] 2023-04-08 22:04:53,731 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-04-08 22:04:53,731 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991493731"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:53,732 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-04-08 22:04:53,733 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:53,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 52 msec 2023-04-08 22:04:53,785 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-04-08 22:04:53,786 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-04-08 22:04:53,805 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 347) - Thread LEAK? -, OpenFileDescriptor=737 (was 656) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=86 (was 86), ProcessCount=163 (was 163), AvailableMemoryMB=4871 (was 4873) 2023-04-08 22:04:53,820 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=737, MaxFileDescriptor=60000, SystemLoadAverage=86, ProcessCount=163, AvailableMemoryMB=4871 2023-04-08 22:04:53,821 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:04:53,822 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:04:53,823 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:04:53,824 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:53,825 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:04:53,826 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:04:53,827 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:53,828 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf empty. 2023-04-08 22:04:53,828 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:53,828 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-04-08 22:04:53,838 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-04-08 22:04:53,840 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 574e7361fc669d4c8cafbcd53e38dfcf, NAME => 'testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 574e7361fc669d4c8cafbcd53e38dfcf, disabling compactions & flushes 2023-04-08 22:04:53,849 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. after waiting 0 ms 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:53,849 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:53,849 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:53,852 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:04:53,853 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991493853"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991493853"}]},"ts":"1680991493853"} 2023-04-08 22:04:53,855 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:04:53,856 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:04:53,856 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991493856"}]},"ts":"1680991493856"} 2023-04-08 22:04:53,857 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-04-08 22:04:53,859 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:04:53,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:04:53,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:04:53,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:04:53,861 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:04:53,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, ASSIGN}] 2023-04-08 22:04:53,863 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, ASSIGN 2023-04-08 22:04:53,863 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:04:54,014 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:04:54,016 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=574e7361fc669d4c8cafbcd53e38dfcf, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,017 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991494016"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991494016"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991494016"}]},"ts":"1680991494016"} 2023-04-08 22:04:54,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:54,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,181 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 574e7361fc669d4c8cafbcd53e38dfcf, NAME => 'testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:04:54,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:04:54,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:54,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,182 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,185 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,187 DEBUG [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1 2023-04-08 22:04:54,187 DEBUG [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1 2023-04-08 22:04:54,188 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 574e7361fc669d4c8cafbcd53e38dfcf columnFamilyName cf1 2023-04-08 22:04:54,189 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] regionserver.HStore(310): Store=574e7361fc669d4c8cafbcd53e38dfcf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:54,189 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,191 DEBUG [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2 2023-04-08 22:04:54,191 DEBUG [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2 2023-04-08 22:04:54,192 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 574e7361fc669d4c8cafbcd53e38dfcf columnFamilyName cf2 2023-04-08 22:04:54,193 INFO [StoreOpener-574e7361fc669d4c8cafbcd53e38dfcf-1] regionserver.HStore(310): Store=574e7361fc669d4c8cafbcd53e38dfcf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:54,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,194 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:04:54,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:54,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:04:54,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 574e7361fc669d4c8cafbcd53e38dfcf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68516258, jitterRate=0.02097180485725403}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:04:54,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:54,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf., pid=111, masterSystemTime=1680991494175 2023-04-08 22:04:54,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,204 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=574e7361fc669d4c8cafbcd53e38dfcf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,204 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991494204"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991494204"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991494204"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991494204"}]},"ts":"1680991494204"} 2023-04-08 22:04:54,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-04-08 22:04:54,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 in 185 msec 2023-04-08 22:04:54,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-04-08 22:04:54,210 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, ASSIGN in 347 msec 2023-04-08 22:04:54,211 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:04:54,212 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991494211"}]},"ts":"1680991494211"} 2023-04-08 22:04:54,213 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-04-08 22:04:54,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:04:54,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 392 msec 2023-04-08 22:04:54,335 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:54,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:54,337 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,337 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,337 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:54,338 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:54,338 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,338 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,339 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,339 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,339 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,339 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,340 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,340 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:54,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,342 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,342 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,342 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:54,342 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,343 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,343 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,343 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,344 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:04:54,347 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,347 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,348 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,348 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,348 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,348 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,356 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/fdfd405d0db94732a443b0564e249755 2023-04-08 22:04:54,376 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/033a7afbabfd4e468ebd756a2ecacc4e 2023-04-08 22:04:54,383 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/fdfd405d0db94732a443b0564e249755 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 2023-04-08 22:04:54,390 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:04:54,391 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/033a7afbabfd4e468ebd756a2ecacc4e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e 2023-04-08 22:04:54,398 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:04:54,399 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 55ms, sequenceid=6, compaction requested=false 2023-04-08 22:04:54,399 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,400 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:54,400 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,400 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:54,400 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:54,400 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,400 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,400 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,402 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,402 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,402 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,402 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:54,406 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@a724a30[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:54,406 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:54,406 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:54,412 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,412 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:54,412 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,413 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,413 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:54,413 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,414 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,414 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,415 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:54,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:54,421 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:54,422 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:54,422 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:54,422 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,422 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,429 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,429 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,430 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,430 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,434 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,434 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,435 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,439 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,439 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,439 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,439 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,440 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,440 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:54,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,541 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:54,543 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:54,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:54,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:54,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:54,565 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:54,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:54,566 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,566 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,567 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,567 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,567 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,567 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,568 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,569 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,569 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,569 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,570 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:54,570 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:54,570 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,570 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:54,570 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:54,581 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,582 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,582 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,582 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,582 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,582 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,583 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/1c57a7e65ff24a3eb52a40228c6ec103 2023-04-08 22:04:54,596 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/e4f715ebb2e749ec90f7deb091ecb56d 2023-04-08 22:04:54,600 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/1c57a7e65ff24a3eb52a40228c6ec103 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 2023-04-08 22:04:54,603 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:04:54,604 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/e4f715ebb2e749ec90f7deb091ecb56d as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d 2023-04-08 22:04:54,607 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:04:54,607 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 37ms, sequenceid=19, compaction requested=false 2023-04-08 22:04:54,607 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:54,607 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,608 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:54,608 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:54,608 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,608 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,608 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,609 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,609 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,609 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,609 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:54,611 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@487e3f2d[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:54,611 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:54,611 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,611 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:54,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,612 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,612 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:54,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,612 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,613 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:54,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:54,615 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:54,615 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:54,615 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:54,615 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,615 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,617 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,617 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,618 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,619 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,619 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,620 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,620 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,621 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,620 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,621 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,721 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:54,723 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:54,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:54,744 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:54,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:54,745 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,745 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,745 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:54,745 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:54,746 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,746 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,747 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,747 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,747 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,747 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,748 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,748 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,748 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,749 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,749 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,750 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,749 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:54,750 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,750 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,750 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,750 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:54,751 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,752 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,752 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,752 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,752 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,752 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,760 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/5cf6c18dce0049648d862a34b441b9a8 2023-04-08 22:04:54,772 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/7f1b174f8196439bb4644eeb814fceb7 2023-04-08 22:04:54,776 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/5cf6c18dce0049648d862a34b441b9a8 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 2023-04-08 22:04:54,781 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:04:54,782 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/7f1b174f8196439bb4644eeb814fceb7 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 2023-04-08 22:04:54,788 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:04:54,789 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 39ms, sequenceid=32, compaction requested=true 2023-04-08 22:04:54,789 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:54,789 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,789 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:54,789 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:54,789 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,789 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,790 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,791 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,791 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,791 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,791 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:54,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4bafd196[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:54,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:54,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,794 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:54,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:54,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:54,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,796 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,796 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:54,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:54,799 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:54,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:54,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:54,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,799 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,800 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,801 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,801 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,802 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,803 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,803 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:54,812 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:54,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,813 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:54,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,814 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,913 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:54,914 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:54,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:54,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:54,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:54,929 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,929 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,929 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:54,929 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:54,930 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,930 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,931 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,931 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,931 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,931 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,932 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,932 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,932 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,932 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,933 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:54,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,934 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:54,934 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:54,934 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:54,935 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:54,935 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:54,935 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,935 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,935 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,935 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:54,935 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:54,936 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,936 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,936 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,936 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,937 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,937 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,945 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/fdbddf36e30644909ca5349402dcb37e 2023-04-08 22:04:54,957 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/9d803c121dfe4cec88e084231f6d8585 2023-04-08 22:04:54,962 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/fdbddf36e30644909ca5349402dcb37e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e 2023-04-08 22:04:54,966 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:04:54,967 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/9d803c121dfe4cec88e084231f6d8585 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 2023-04-08 22:04:54,970 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:04:54,970 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 35ms, sequenceid=45, compaction requested=true 2023-04-08 22:04:54,970 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:54,970 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:54,970 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:54,971 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:54,971 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,971 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:54,971 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:54,972 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,972 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,972 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,972 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:54,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:54,974 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@165bd797[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:54,974 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:54,974 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,975 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,975 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:54,975 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,976 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,976 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:54,979 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:54,979 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:54,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:54,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:54,979 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:54,979 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,980 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,980 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,980 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,981 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,983 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:54,983 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:54,983 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,985 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,985 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,985 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,986 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:54,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,985 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:54,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:54,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:54,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:54,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:54,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,087 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:55,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:55,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:55,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:55,111 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:55,111 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:55,111 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,111 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:55,111 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:55,112 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,112 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,113 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,113 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,113 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,113 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,113 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,114 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,115 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,114 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,115 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:55,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:55,115 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:55,115 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:55,115 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,117 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,124 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/27c786f359ef4264950d621f87a0198b 2023-04-08 22:04:55,138 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/862142fe15004bb7a45aeab624443d3d 2023-04-08 22:04:55,142 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/27c786f359ef4264950d621f87a0198b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b 2023-04-08 22:04:55,146 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:04:55,146 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/862142fe15004bb7a45aeab624443d3d as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d 2023-04-08 22:04:55,150 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:04:55,151 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 36ms, sequenceid=58, compaction requested=true 2023-04-08 22:04:55,151 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:55,151 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:55,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:55,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:55,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,151 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,153 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,153 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,153 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,153 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,155 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:55,156 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@e602ca6[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:55,156 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:55,156 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,156 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,156 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,156 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,156 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,157 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,157 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:55,157 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,158 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,158 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,159 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:55,162 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:55,162 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:55,162 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:55,162 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:55,162 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,163 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,164 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,164 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,164 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,165 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,165 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,167 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,168 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,168 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:55,171 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,171 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:55,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:55,171 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,172 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,172 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,272 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:55,273 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:55,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:04:55,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-04-08 22:04:55,289 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-04-08 22:04:55,289 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:55,289 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,289 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-04-08 22:04:55,289 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-04-08 22:04:55,289 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,289 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,290 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,290 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,290 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,290 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,291 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,291 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-04-08 22:04:55,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,293 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,293 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-04-08 22:04:55,293 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,293 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. started... 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:04:55,293 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,294 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-04-08 22:04:55,294 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:04:55,294 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,294 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,295 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,308 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/ae411d9edd2c4d3683b317e9d1845a48 2023-04-08 22:04:55,318 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/a0627e7879334c8bab18933f8ca0dc1e 2023-04-08 22:04:55,322 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/ae411d9edd2c4d3683b317e9d1845a48 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 2023-04-08 22:04:55,326 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:04:55,326 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/a0627e7879334c8bab18933f8ca0dc1e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e 2023-04-08 22:04:55,330 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:04:55,331 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 37ms, sequenceid=71, compaction requested=true 2023-04-08 22:04:55,331 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:55,331 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:55,331 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:04:55,331 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:04:55,331 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,331 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-04-08 22:04:55,331 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-04-08 22:04:55,332 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,332 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,332 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,333 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-04-08 22:04:55,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@375eed10[Count = 0] remaining members to acquire global barrier 2023-04-08 22:04:55,334 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-04-08 22:04:55,334 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,334 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,334 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,335 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,335 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:04:55,334 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:04:55,335 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,336 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,336 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,336 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,336 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,336 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,336 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,337 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-04-08 22:04:55,337 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,337 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:04:55,339 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-04-08 22:04:55,339 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-04-08 22:04:55,339 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:04:55,339 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:04:55,339 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-04-08 22:04:55,339 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:04:55,340 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,340 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,340 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:04:55,341 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:04:55,342 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-04-08 22:04:55,342 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,342 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-04-08 22:04:55,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:04:55,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,344 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:04:55,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:04:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:04:55,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-04-08 22:04:55,344 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:04:55,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:04:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,345 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-04-08 22:04:55,445 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:04:55,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:04:55,714 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-04-08 22:04:55,716 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:55,718 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,721 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:55,723 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,723 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:55,723 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:55,969 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,969 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:55,969 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:55,969 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:55,970 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-04-08 22:04:55,970 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:55,970 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:55,970 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:55,970 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:55,972 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:04:55,975 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-04-08 22:04:55,976 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:55,976 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:55,977 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:55,977 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:55,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:55,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:55,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:55,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:55,978 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:55,979 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:55,980 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:55,981 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:55,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:55,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:55,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:55,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,982 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:55,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,982 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,982 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-04-08 22:04:55,983 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:55,984 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:55,984 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991525984 2023-04-08 22:04:55,984 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:55,986 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,987 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:55,988 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:55,988 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:55,988 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991555988 2023-04-08 22:04:55,988 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991615988 2023-04-08 22:04:55,988 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:55,988 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:55,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:55,990 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:55,990 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:55,990 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:56,238 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,238 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:56,238 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:56,238 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:56,240 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:56,240 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:56,240 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:56,240 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:56,240 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:56,241 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:56,246 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-04-08 22:04:56,246 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:56,247 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:56,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:56,248 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:56,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:56,249 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,250 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,250 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:56,251 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,252 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,252 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,253 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:56,253 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:56,253 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,253 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,254 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,255 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,256 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,256 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:56,256 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,256 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:56,256 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,256 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,256 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,256 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-04-08 22:04:56,257 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:56,257 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:56,258 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991526258 2023-04-08 22:04:56,259 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,259 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:56,261 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:56,261 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:56,262 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:56,262 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991556262 2023-04-08 22:04:56,262 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991616262 2023-04-08 22:04:56,262 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:04:56,262 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:56,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN}] 2023-04-08 22:04:56,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:56,264 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,264 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:56,264 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:56,512 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,512 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:56,512 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:56,512 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:56,512 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:56,513 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:56,513 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:56,513 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:56,513 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:56,513 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:56,515 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-04-08 22:04:56,515 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:56,515 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:56,515 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:56,515 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,516 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:56,517 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,517 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,517 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,518 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:56,518 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:56,518 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,519 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,519 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,519 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:56,519 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,520 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,520 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,520 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:56,521 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,521 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,521 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,521 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,522 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:56,522 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,522 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,523 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,523 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,523 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,523 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:56,524 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,524 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:56,524 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,524 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,524 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,524 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-04-08 22:04:56,525 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:56,526 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:56,526 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991526526 2023-04-08 22:04:56,527 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:56,527 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,530 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:56,530 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:56,530 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:56,530 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991556530 2023-04-08 22:04:56,530 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991616530 2023-04-08 22:04:56,531 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:56,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN}] 2023-04-08 22:04:56,531 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:56,533 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:56,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:56,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991496533"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991496533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991496533"}]},"ts":"1680991496533"} 2023-04-08 22:04:56,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:56,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 574e7361fc669d4c8cafbcd53e38dfcf, disabling compactions & flushes 2023-04-08 22:04:56,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. after waiting 0 ms 2023-04-08 22:04:56,687 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:56,687 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 574e7361fc669d4c8cafbcd53e38dfcf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:04:56,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/37ded22c26df4d2baab7fb6eb1cfa77f 2023-04-08 22:04:56,719 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/02e0198af3c04f8aaa4db8c12453d39b 2023-04-08 22:04:56,724 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf1/37ded22c26df4d2baab7fb6eb1cfa77f as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f 2023-04-08 22:04:56,729 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:04:56,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/.tmp/cf2/02e0198af3c04f8aaa4db8c12453d39b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b 2023-04-08 22:04:56,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:04:56,733 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 574e7361fc669d4c8cafbcd53e38dfcf in 46ms, sequenceid=82, compaction requested=true 2023-04-08 22:04:56,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:04:56,737 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:56,738 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. 2023-04-08 22:04:56,738 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 574e7361fc669d4c8cafbcd53e38dfcf: 2023-04-08 22:04:56,740 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED 2023-04-08 22:04:56,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991496741"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991496741"}]},"ts":"1680991496741"} 2023-04-08 22:04:56,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-04-08 22:04:56,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 in 208 msec 2023-04-08 22:04:56,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-04-08 22:04:56,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN in 212 msec 2023-04-08 22:04:56,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:56,747 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:56,747 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:56,781 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:56,781 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:56,782 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:56,783 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-04-08 22:04:56,783 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:56,783 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:56,783 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:56,783 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,784 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,785 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,786 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,787 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:56,788 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,788 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,788 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:56,788 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:56,788 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-04-08 22:04:56,789 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:56,790 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:56,790 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991526790 2023-04-08 22:04:56,790 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:56,791 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:56,793 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:56,793 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:56,793 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:56,794 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991556794 2023-04-08 22:04:56,794 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991616794 2023-04-08 22:04:56,794 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:56,796 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:56,806 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=574e7361fc669d4c8cafbcd53e38dfcf, threads=14 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,807 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,808 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,808 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,808 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,808 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,959 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,969 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,970 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,970 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,975 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,981 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:04:56,984 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,987 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,987 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,988 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,990 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,991 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,992 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,993 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,994 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:56,994 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 574e7361fc669d4c8cafbcd53e38dfcf Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:04:57,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:57,022 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:57,022 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:57,044 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:57,044 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:57,045 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:57,045 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:57,046 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-04-08 22:04:57,046 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:57,046 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:57,046 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,047 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:57,048 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,049 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,050 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:57,050 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,050 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,050 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,050 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,051 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:57,051 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,051 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,051 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,051 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:57,052 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:57,053 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,053 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,053 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,053 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,053 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,053 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-04-08 22:04:57,054 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:57,054 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:57,055 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991527055 2023-04-08 22:04:57,056 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,056 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:57,058 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:57,058 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:57,058 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:57,059 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991557059 2023-04-08 22:04:57,059 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991617059 2023-04-08 22:04:57,059 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:04:57,066 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=574e7361fc669d4c8cafbcd53e38dfcf, threads=14 2023-04-08 22:04:57,066 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,066 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,066 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,066 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,066 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,067 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,067 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,067 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,067 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,073 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,073 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,073 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,073 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,073 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,140 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,152 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,154 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,155 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,156 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,158 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,160 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,161 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,162 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,162 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,163 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,164 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,164 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,165 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f for region: 574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,165 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region 574e7361fc669d4c8cafbcd53e38dfcf Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:04:57,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:57,205 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:04:57,208 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:04:57,209 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:57,209 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:57,209 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:57,309 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,309 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:57,310 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:57,310 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:57,310 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:57,311 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:57,311 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:57,311 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:57,311 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:57,313 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:04:57,318 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-04-08 22:04:57,319 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:57,319 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:57,319 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:57,319 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:57,319 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:57,320 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:57,320 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,320 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:57,320 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,320 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,321 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,321 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:57,321 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,321 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,321 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,322 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:57,322 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:57,322 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,322 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,322 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,323 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:57,323 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,323 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,323 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,323 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:57,324 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,324 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,324 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,324 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,325 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,326 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:57,326 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,326 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:57,326 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:57,326 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:57,326 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,327 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,327 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,327 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,327 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,327 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 13 msec 2023-04-08 22:04:57,328 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:57,328 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:57,328 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991527328 2023-04-08 22:04:57,329 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:57,330 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,332 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:57,332 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:57,332 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:57,332 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991557332 2023-04-08 22:04:57,332 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991617332 2023-04-08 22:04:57,332 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:57,335 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:57,336 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:57,336 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,337 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:57,337 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:57,582 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,583 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:57,583 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:57,583 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:57,584 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:57,584 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:57,584 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:57,584 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:57,585 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:57,586 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:04:57,590 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-04-08 22:04:57,590 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:57,590 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:57,591 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:57,592 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:57,592 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:57,592 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:57,593 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,593 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,594 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,595 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,596 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:57,597 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:57,598 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,598 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,598 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,598 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,598 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,598 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-04-08 22:04:57,599 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:57,599 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:57,600 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991527600 2023-04-08 22:04:57,601 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:57,601 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,603 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:57,603 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:57,603 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:57,603 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991557603 2023-04-08 22:04:57,603 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991617603 2023-04-08 22:04:57,603 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-04-08 22:04:57,604 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:57,605 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991497605"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1680991497605"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1680991497605"}]},"ts":"1680991497605"} 2023-04-08 22:04:57,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991497605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991497605"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991497605"}]},"ts":"1680991497605"} 2023-04-08 22:04:57,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991497605"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991497605"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991497605"}]},"ts":"1680991497605"} 2023-04-08 22:04:57,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:57,607 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:57,607 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:57,854 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,854 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:57,854 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:57,855 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:57,855 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:57,856 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:57,856 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:57,856 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:57,856 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:57,858 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:04:57,861 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-04-08 22:04:57,862 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:57,862 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:57,862 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:57,863 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:57,864 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:57,864 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:57,865 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,865 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,866 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:57,867 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,867 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:57,867 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:57,867 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,867 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:57,868 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,869 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:57,869 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,869 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,869 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:57,869 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:57,870 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:57,871 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,871 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,871 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,871 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,871 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:57,871 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-04-08 22:04:57,872 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:57,872 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:57,872 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991527872 2023-04-08 22:04:57,873 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:57,874 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,876 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:57,876 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:57,876 INFO [Listener at localhost.localdomain/46567] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002, table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:57,876 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=8a43fb69950c4167727f2a330ede3bf4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:57,877 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=12baaa72406c0bdceee750f3da4ff69d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:57,877 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:57,877 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991557877 2023-04-08 22:04:57,877 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991617877 2023-04-08 22:04:57,877 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:57,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1680991497877"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1680991497877"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1680991497877"}]},"ts":"1680991497877"} 2023-04-08 22:04:57,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991497877"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991497877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991497877"}]},"ts":"1680991497877"} 2023-04-08 22:04:57,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991497877"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991497877"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991497877"}]},"ts":"1680991497877"} 2023-04-08 22:04:57,879 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:57,881 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:57,881 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:57,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:57,881 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:58,127 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,127 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:58,127 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:58,128 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:58,128 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:58,128 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:58,128 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:58,128 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:58,128 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:58,129 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:58,130 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-04-08 22:04:58,130 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:58,130 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:58,130 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:58,131 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:58,131 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:58,131 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:58,131 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,131 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,132 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,133 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,133 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:58,133 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:58,133 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,133 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,134 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:58,135 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,136 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,136 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,136 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,136 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,136 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:58,137 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,137 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,137 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:58,137 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:58,137 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-04-08 22:04:58,138 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:58,139 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:58,139 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991528139 2023-04-08 22:04:58,139 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:58,141 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,143 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:58,143 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:58,143 INFO [Listener at localhost.localdomain/46567] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002, table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:58,143 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=8a43fb69950c4167727f2a330ede3bf4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:58,143 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=12baaa72406c0bdceee750f3da4ff69d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:58,143 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:58,144 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991558144 2023-04-08 22:04:58,144 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991618144 2023-04-08 22:04:58,144 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-04-08 22:04:58,144 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:58,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN}] 2023-04-08 22:04:58,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:58,146 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,146 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:58,146 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:58,394 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,394 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:58,394 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:58,395 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:58,395 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:58,395 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:58,396 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:58,396 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:58,396 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:58,397 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-04-08 22:04:58,399 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-04-08 22:04:58,400 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:58,400 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:58,400 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:58,401 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:58,401 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:58,401 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:58,402 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,402 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:58,402 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,403 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:58,403 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,403 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:58,403 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,404 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:58,404 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,404 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:58,405 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:58,405 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,405 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:58,406 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,406 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:58,406 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,406 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:58,407 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,407 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:58,407 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,407 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:58,408 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,408 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,409 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:58,410 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,410 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,410 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:58,410 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:58,410 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-04-08 22:04:58,411 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:58,412 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:58,412 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991528412 2023-04-08 22:04:58,413 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:58,413 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,416 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:58,416 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:58,416 INFO [Listener at localhost.localdomain/46567] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002, table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:58,416 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=8a43fb69950c4167727f2a330ede3bf4, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:58,416 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=12baaa72406c0bdceee750f3da4ff69d, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-04-08 22:04:58,416 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:58,416 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991558416 2023-04-08 22:04:58,416 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991618416 2023-04-08 22:04:58,416 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-04-08 22:04:58,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN}] 2023-04-08 22:04:58,417 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:58,418 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN 2023-04-08 22:04:58,419 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:04:58,419 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN 2023-04-08 22:04:58,420 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN; state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:04:58,569 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:04:58,572 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=12baaa72406c0bdceee750f3da4ff69d, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,572 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991498571"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991498571"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991498571"}]},"ts":"1680991498571"} 2023-04-08 22:04:58,574 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=116, state=RUNNABLE; OpenRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:58,578 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=8a43fb69950c4167727f2a330ede3bf4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,579 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991498578"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991498578"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991498578"}]},"ts":"1680991498578"} 2023-04-08 22:04:58,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=115, state=RUNNABLE; OpenRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:58,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 12baaa72406c0bdceee750f3da4ff69d, NAME => 'testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.', STARTKEY => '41', ENDKEY => ''} 2023-04-08 22:04:58,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:04:58,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:58,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,737 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,739 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1 2023-04-08 22:04:58,739 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1 2023-04-08 22:04:58,740 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12baaa72406c0bdceee750f3da4ff69d columnFamilyName cf1 2023-04-08 22:04:58,755 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-top 2023-04-08 22:04:58,760 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-top 2023-04-08 22:04:58,765 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-top 2023-04-08 22:04:58,769 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-top 2023-04-08 22:04:58,773 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-top 2023-04-08 22:04:58,777 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-top 2023-04-08 22:04:58,781 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-top 2023-04-08 22:04:58,782 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(310): Store=12baaa72406c0bdceee750f3da4ff69d/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:58,782 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,783 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2 2023-04-08 22:04:58,783 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2 2023-04-08 22:04:58,783 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 12baaa72406c0bdceee750f3da4ff69d columnFamilyName cf2 2023-04-08 22:04:58,797 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-top 2023-04-08 22:04:58,803 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-top 2023-04-08 22:04:58,808 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-top 2023-04-08 22:04:58,812 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-top 2023-04-08 22:04:58,816 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-top 2023-04-08 22:04:58,820 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-top 2023-04-08 22:04:58,825 DEBUG [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-top 2023-04-08 22:04:58,826 INFO [StoreOpener-12baaa72406c0bdceee750f3da4ff69d-1] regionserver.HStore(310): Store=12baaa72406c0bdceee750f3da4ff69d/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:58,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,827 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,829 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:04:58,830 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 12baaa72406c0bdceee750f3da4ff69d; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71720525, jitterRate=0.06871910393238068}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:04:58,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 12baaa72406c0bdceee750f3da4ff69d: 2023-04-08 22:04:58,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d., pid=117, masterSystemTime=1680991498727 2023-04-08 22:04:58,831 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:04:58,833 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:04:58,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:58,834 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:04:58,834 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,834 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,834 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): 12baaa72406c0bdceee750f3da4ff69d/cf1 is initiating minor compaction (all files) 2023-04-08 22:04:58,834 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): 12baaa72406c0bdceee750f3da4ff69d/cf2 is initiating minor compaction (all files) 2023-04-08 22:04:58,834 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 12baaa72406c0bdceee750f3da4ff69d/cf1 in testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,834 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 12baaa72406c0bdceee750f3da4ff69d/cf2 in testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,835 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/.tmp, totalSize=34.9 K 2023-04-08 22:04:58,835 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/.tmp, totalSize=34.9 K 2023-04-08 22:04:58,835 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991494327 2023-04-08 22:04:58,835 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991494327 2023-04-08 22:04:58,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:58,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,835 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991494544 2023-04-08 22:04:58,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8a43fb69950c4167727f2a330ede3bf4, NAME => 'testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.', STARTKEY => '', ENDKEY => '41'} 2023-04-08 22:04:58,836 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=12baaa72406c0bdceee750f3da4ff69d, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,836 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991494544 2023-04-08 22:04:58,836 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991498835"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991498835"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991498835"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991498835"}]},"ts":"1680991498835"} 2023-04-08 22:04:58,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:04:58,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:04:58,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,836 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991494724 2023-04-08 22:04:58,837 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991494724 2023-04-08 22:04:58,837 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991494914 2023-04-08 22:04:58,837 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991494914 2023-04-08 22:04:58,837 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,837 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991495089 2023-04-08 22:04:58,838 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991495089 2023-04-08 22:04:58,838 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991495274 2023-04-08 22:04:58,838 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991495274 2023-04-08 22:04:58,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=116 2023-04-08 22:04:58,838 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1 2023-04-08 22:04:58,838 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1 2023-04-08 22:04:58,838 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002 in 263 msec 2023-04-08 22:04:58,838 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991495449 2023-04-08 22:04:58,838 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991495449 2023-04-08 22:04:58,838 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a43fb69950c4167727f2a330ede3bf4 columnFamilyName cf1 2023-04-08 22:04:58,839 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN in 421 msec 2023-04-08 22:04:58,847 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): 12baaa72406c0bdceee750f3da4ff69d#cf1#compaction#106 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:04:58,847 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): 12baaa72406c0bdceee750f3da4ff69d#cf2#compaction#107 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:04:58,852 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-bottom 2023-04-08 22:04:58,859 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-bottom 2023-04-08 22:04:58,869 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-bottom 2023-04-08 22:04:58,871 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/.tmp/cf2/974d02517dc3478c8964b3d3a91e7f3a as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/974d02517dc3478c8964b3d3a91e7f3a 2023-04-08 22:04:58,875 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 12baaa72406c0bdceee750f3da4ff69d/cf2 of 12baaa72406c0bdceee750f3da4ff69d into 974d02517dc3478c8964b3d3a91e7f3a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:04:58,875 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 12baaa72406c0bdceee750f3da4ff69d: 2023-04-08 22:04:58,875 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d., storeName=12baaa72406c0bdceee750f3da4ff69d/cf2, priority=9, startTime=1680991498833; duration=0sec 2023-04-08 22:04:58,875 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:58,876 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-bottom 2023-04-08 22:04:58,876 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/.tmp/cf1/4c98cc28340d485e8478c9f84b400042 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/4c98cc28340d485e8478c9f84b400042 2023-04-08 22:04:58,879 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-bottom 2023-04-08 22:04:58,880 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 12baaa72406c0bdceee750f3da4ff69d/cf1 of 12baaa72406c0bdceee750f3da4ff69d into 4c98cc28340d485e8478c9f84b400042(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:04:58,880 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 12baaa72406c0bdceee750f3da4ff69d: 2023-04-08 22:04:58,881 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d., storeName=12baaa72406c0bdceee750f3da4ff69d/cf1, priority=9, startTime=1680991498831; duration=0sec 2023-04-08 22:04:58,881 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:58,882 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-bottom 2023-04-08 22:04:58,886 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-bottom 2023-04-08 22:04:58,886 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(310): Store=8a43fb69950c4167727f2a330ede3bf4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:58,886 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,887 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2 2023-04-08 22:04:58,887 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2 2023-04-08 22:04:58,887 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8a43fb69950c4167727f2a330ede3bf4 columnFamilyName cf2 2023-04-08 22:04:58,900 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-bottom 2023-04-08 22:04:58,903 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-bottom 2023-04-08 22:04:58,906 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-bottom 2023-04-08 22:04:58,909 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-bottom 2023-04-08 22:04:58,912 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-bottom 2023-04-08 22:04:58,915 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-bottom 2023-04-08 22:04:58,918 DEBUG [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-bottom 2023-04-08 22:04:58,919 INFO [StoreOpener-8a43fb69950c4167727f2a330ede3bf4-1] regionserver.HStore(310): Store=8a43fb69950c4167727f2a330ede3bf4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:04:58,919 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,920 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,921 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:04:58,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:58,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 8a43fb69950c4167727f2a330ede3bf4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70185435, jitterRate=0.04584448039531708}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:04:58,923 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 8a43fb69950c4167727f2a330ede3bf4: 2023-04-08 22:04:58,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4., pid=118, masterSystemTime=1680991498727 2023-04-08 22:04:58,924 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:58,925 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:04:58,925 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:04:58,925 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,925 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): 8a43fb69950c4167727f2a330ede3bf4/cf2 is initiating minor compaction (all files) 2023-04-08 22:04:58,925 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): 8a43fb69950c4167727f2a330ede3bf4/cf1 is initiating minor compaction (all files) 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 8a43fb69950c4167727f2a330ede3bf4/cf2 in testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8a43fb69950c4167727f2a330ede3bf4/cf1 in testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/.tmp, totalSize=34.9 K 2023-04-08 22:04:58,925 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/.tmp, totalSize=34.9 K 2023-04-08 22:04:58,926 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991494327 2023-04-08 22:04:58,926 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991494327 2023-04-08 22:04:58,926 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:58,926 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991494544 2023-04-08 22:04:58,926 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991494544 2023-04-08 22:04:58,926 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=8a43fb69950c4167727f2a330ede3bf4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991494724 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991494724 2023-04-08 22:04:58,927 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991498926"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991498926"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991498926"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991498926"}]},"ts":"1680991498926"} 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991494914 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991494914 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991495089 2023-04-08 22:04:58,927 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991495089 2023-04-08 22:04:58,928 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991495274 2023-04-08 22:04:58,928 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991495274 2023-04-08 22:04:58,928 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991495449 2023-04-08 22:04:58,928 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991495449 2023-04-08 22:04:58,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=115 2023-04-08 22:04:58,929 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002 in 347 msec 2023-04-08 22:04:58,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-04-08 22:04:58,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN in 512 msec 2023-04-08 22:04:58,930 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-04-08 22:04:58,930 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:58,930 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:04:58,930 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-04-08 22:04:58,937 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): 8a43fb69950c4167727f2a330ede3bf4#cf2#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:04:58,937 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8a43fb69950c4167727f2a330ede3bf4#cf1#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:04:58,948 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/.tmp/cf2/1e29fee01e3947ce818a79252648f47f as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/1e29fee01e3947ce818a79252648f47f 2023-04-08 22:04:58,948 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/.tmp/cf1/7c2df3e0b95a4f8db65abff9b278100c as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/7c2df3e0b95a4f8db65abff9b278100c 2023-04-08 22:04:58,952 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a43fb69950c4167727f2a330ede3bf4/cf1 of 8a43fb69950c4167727f2a330ede3bf4 into 7c2df3e0b95a4f8db65abff9b278100c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:04:58,952 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8a43fb69950c4167727f2a330ede3bf4/cf2 of 8a43fb69950c4167727f2a330ede3bf4 into 1e29fee01e3947ce818a79252648f47f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:04:58,952 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8a43fb69950c4167727f2a330ede3bf4: 2023-04-08 22:04:58,952 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8a43fb69950c4167727f2a330ede3bf4: 2023-04-08 22:04:58,952 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4., storeName=8a43fb69950c4167727f2a330ede3bf4/cf1, priority=9, startTime=1680991498924; duration=0sec 2023-04-08 22:04:58,952 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4., storeName=8a43fb69950c4167727f2a330ede3bf4/cf2, priority=9, startTime=1680991498925; duration=0sec 2023-04-08 22:04:58,953 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:58,953 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:04:59,167 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,167 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-04-08 22:04:59,167 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:04:59,167 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:04:59,168 ERROR [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-04-08 22:04:59,168 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-04-08 22:04:59,168 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-04-08 22:04:59,168 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-04-08 22:04:59,168 INFO [Listener at localhost.localdomain/46567] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-04-08 22:04:59,169 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-04-08 22:04:59,170 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-04-08 22:04:59,170 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-04-08 22:04:59,170 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 18, 23] 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 16, 21] 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-04-08 22:04:59,171 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-04-08 22:04:59,172 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-04-08 22:04:59,172 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-04-08 22:04:59,172 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:59,172 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=9ad6c05dacbf0ddb8d22dd1a8d21f5f2, daughterA=96ecd1c4fcf2c0e9e4913ef6a785f1b7, daughterB=906521a2f5315a45a80af68b91b05565 2023-04-08 22:04:59,172 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:59,173 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-04-08 22:04:59,173 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:59,173 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6f9219e0fb8fc810d2587ec30edba29e, daughterA=f059ab2aed67beb6b09f8d73a44d767b, daughterB=68c1c7892be17242d3b4efe192ab38a1 2023-04-08 22:04:59,173 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:59,173 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-04-08 22:04:59,174 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:59,174 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=53819c752f224f75f5ed122f97922705, daughterA=73ae6ed38cae9b85995205c0a1b1aeb0, daughterB=85c4a4ed50826801b0ff23921e269eb9 2023-04-08 22:04:59,174 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=53819c752f224f75f5ed122f97922705, ASSIGN 2023-04-08 22:04:59,174 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:59,174 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-04-08 22:04:59,175 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:59,175 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=18826054628d89f6df3d6e684f01e947, daughterA=861bb60d37055f19de7791db1f5f8627, daughterB=cb76bcec70ebbefd8b284e846da96f4e 2023-04-08 22:04:59,175 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:59,175 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-04-08 22:04:59,175 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:59,176 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ee9b8d200835a543c2b4c8142e56c58a, daughterA=cfac3b4c98d7f2cc5feb51a9996121dc, daughterB=ae1a4e6d25e5fb5dd5133abebab5ecb7 2023-04-08 22:04:59,176 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:59,176 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-04-08 22:04:59,176 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:59,176 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase20.apache.org,43551,1680991397921, table=testRollbackForSplitTableRegionWithReplica, region=33a379c734056336fa36cffbc96b48b1 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=2e5e8a708bb5782faafc9eb08d64e11f, daughterA=127e8d985700fa7e97d23a14c409fe15, daughterB=a8f0a611ff2cfb6d1108d87d54baed99 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:59,177 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-04-08 22:04:59,178 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN 2023-04-08 22:04:59,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN 2023-04-08 22:04:59,179 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d held the lock before restarting, call acquireLock to restore it. 2023-04-08 22:04:59,180 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,180 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,180 INFO [Listener at localhost.localdomain/46567] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure 574e7361fc669d4c8cafbcd53e38dfcf, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,180 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,181 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=117, ppid=116, state=SUCCESS; OpenRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,181 DEBUG [Listener at localhost.localdomain/46567] procedure2.Procedure(946): pid=118, ppid=115, state=SUCCESS; OpenRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002 didn't hold the lock before restarting, skip acquiring lock. 2023-04-08 22:04:59,181 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-04-08 22:04:59,182 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-04-08 22:04:59,182 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967, table=hbase:meta, region=1588230740 2023-04-08 22:04:59,183 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1680991529183 2023-04-08 22:04:59,183 DEBUG [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1517): Joining cluster... 2023-04-08 22:04:59,184 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,186 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=b6167fccb8f7fec5f3eb188ccc6fd56e, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41749,1680991397967, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967, openSeqNum=2 2023-04-08 22:04:59,187 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=574e7361fc669d4c8cafbcd53e38dfcf, regionState=CLOSED, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=2 2023-04-08 22:04:59,187 INFO [Listener at localhost.localdomain/46567] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase20.apache.org,41779,1680991398002, table=testRecoveryAndDoubleExecution, region=574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,187 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=8a43fb69950c4167727f2a330ede3bf4, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=86 2023-04-08 22:04:59,187 DEBUG [Listener at localhost.localdomain/46567] assignment.RegionStateStore(147): Load hbase:meta entry region=12baaa72406c0bdceee750f3da4ff69d, regionState=OPEN, lastHost=jenkins-hbase20.apache.org,41779,1680991398002, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002, openSeqNum=86 2023-04-08 22:04:59,188 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-04-08 22:04:59,188 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1680991559188 2023-04-08 22:04:59,188 INFO [Listener at localhost.localdomain/46567] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1680991619188 2023-04-08 22:04:59,188 INFO [Listener at localhost.localdomain/46567] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-04-08 22:04:59,188 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-04-08 22:04:59,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=574e7361fc669d4c8cafbcd53e38dfcf, daughterA=8a43fb69950c4167727f2a330ede3bf4, daughterB=12baaa72406c0bdceee750f3da4ff69d in 3.4720 sec 2023-04-08 22:04:59,439 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41779] ipc.CallRunner(144): callId: 888 service: ClientService methodName: Scan size: 154 connection: 148.251.75.209:32990 deadline: 1680991559439, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf. is not online on jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,551 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:04:59,562 INFO [Listener at localhost.localdomain/46567] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-08 22:04:59,564 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:04:59,574 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:04:59,576 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-04-08 22:04:59,576 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testRecoveryAndDoubleExecution 2023-04-08 22:04:59,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,579 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991499579"}]},"ts":"1680991499579"} 2023-04-08 22:04:59,579 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-08 22:04:59,580 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-04-08 22:04:59,581 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-04-08 22:04:59,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, UNASSIGN}] 2023-04-08 22:04:59,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, UNASSIGN 2023-04-08 22:04:59,582 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=12baaa72406c0bdceee750f3da4ff69d, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,582 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991499582"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991499582"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991499582"}]},"ts":"1680991499582"} 2023-04-08 22:04:59,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:59,585 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, UNASSIGN 2023-04-08 22:04:59,585 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=8a43fb69950c4167727f2a330ede3bf4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:04:59,585 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991499585"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991499585"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991499585"}]},"ts":"1680991499585"} 2023-04-08 22:04:59,586 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:04:59,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-08 22:04:59,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:59,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 8a43fb69950c4167727f2a330ede3bf4, disabling compactions & flushes 2023-04-08 22:04:59,735 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:59,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:59,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. after waiting 0 ms 2023-04-08 22:04:59,735 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:59,736 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-bottom] to archive 2023-04-08 22:04:59,738 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:04:59,742 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,745 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,747 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,749 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,750 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,752 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,753 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,755 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-bottom] to archive 2023-04-08 22:04:59,756 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:04:59,757 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,759 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,760 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,761 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,763 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,764 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,765 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,770 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:04:59,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:59,771 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4. 2023-04-08 22:04:59,771 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 8a43fb69950c4167727f2a330ede3bf4: 2023-04-08 22:04:59,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:59,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 12baaa72406c0bdceee750f3da4ff69d, disabling compactions & flushes 2023-04-08 22:04:59,773 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:59,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:59,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. after waiting 0 ms 2023-04-08 22:04:59,773 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:59,773 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=8a43fb69950c4167727f2a330ede3bf4, regionState=CLOSED 2023-04-08 22:04:59,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991499773"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991499773"}]},"ts":"1680991499773"} 2023-04-08 22:04:59,775 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f-top] to archive 2023-04-08 22:04:59,775 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:04:59,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-04-08 22:04:59,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 8a43fb69950c4167727f2a330ede3bf4, server=jenkins-hbase20.apache.org,41779,1680991398002 in 189 msec 2023-04-08 22:04:59,776 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=8a43fb69950c4167727f2a330ede3bf4, UNASSIGN in 195 msec 2023-04-08 22:04:59,777 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,778 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,779 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,780 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,780 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,781 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,782 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b-top] to archive 2023-04-08 22:04:59,783 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:04:59,784 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,785 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,786 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,786 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,787 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,788 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,789 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:04:59,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:04:59,792 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d. 2023-04-08 22:04:59,792 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 12baaa72406c0bdceee750f3da4ff69d: 2023-04-08 22:04:59,794 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,794 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=12baaa72406c0bdceee750f3da4ff69d, regionState=CLOSED 2023-04-08 22:04:59,794 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1680991499794"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991499794"}]},"ts":"1680991499794"} 2023-04-08 22:04:59,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-04-08 22:04:59,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 12baaa72406c0bdceee750f3da4ff69d, server=jenkins-hbase20.apache.org,41779,1680991398002 in 212 msec 2023-04-08 22:04:59,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-04-08 22:04:59,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=12baaa72406c0bdceee750f3da4ff69d, UNASSIGN in 216 msec 2023-04-08 22:04:59,799 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991499798"}]},"ts":"1680991499798"} 2023-04-08 22:04:59,799 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-04-08 22:04:59,800 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-04-08 22:04:59,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 225 msec 2023-04-08 22:04:59,882 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-04-08 22:04:59,883 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-04-08 22:04:59,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testRecoveryAndDoubleExecution 2023-04-08 22:04:59,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,890 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,891 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-04-08 22:04:59,898 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,898 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:59,898 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,901 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/recovered.edits] 2023-04-08 22:04:59,902 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/recovered.edits] 2023-04-08 22:04:59,902 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/recovered.edits] 2023-04-08 22:04:59,906 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/.regioninfo to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/.regioninfo 2023-04-08 22:04:59,906 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/.regioninfo to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/.regioninfo 2023-04-08 22:04:59,907 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/1c57a7e65ff24a3eb52a40228c6ec103 2023-04-08 22:04:59,909 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/27c786f359ef4264950d621f87a0198b 2023-04-08 22:04:59,909 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,910 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/1c57a7e65ff24a3eb52a40228c6ec103.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,911 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/37ded22c26df4d2baab7fb6eb1cfa77f 2023-04-08 22:04:59,911 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,911 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/27c786f359ef4264950d621f87a0198b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,912 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/5cf6c18dce0049648d862a34b441b9a8 2023-04-08 22:04:59,913 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,913 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/37ded22c26df4d2baab7fb6eb1cfa77f.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,914 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/ae411d9edd2c4d3683b317e9d1845a48 2023-04-08 22:04:59,914 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,914 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/5cf6c18dce0049648d862a34b441b9a8.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,915 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdbddf36e30644909ca5349402dcb37e 2023-04-08 22:04:59,916 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,916 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/ae411d9edd2c4d3683b317e9d1845a48.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,917 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf1/fdfd405d0db94732a443b0564e249755 2023-04-08 22:04:59,917 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,917 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/fdbddf36e30644909ca5349402dcb37e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,919 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,919 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf1/fdfd405d0db94732a443b0564e249755.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,920 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/02e0198af3c04f8aaa4db8c12453d39b 2023-04-08 22:04:59,921 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/033a7afbabfd4e468ebd756a2ecacc4e 2023-04-08 22:04:59,921 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,921 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/02e0198af3c04f8aaa4db8c12453d39b.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,923 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,923 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/033a7afbabfd4e468ebd756a2ecacc4e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,923 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/7f1b174f8196439bb4644eeb814fceb7 2023-04-08 22:04:59,924 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,924 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/862142fe15004bb7a45aeab624443d3d 2023-04-08 22:04:59,925 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,925 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/9d803c121dfe4cec88e084231f6d8585 2023-04-08 22:04:59,926 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,926 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/a0627e7879334c8bab18933f8ca0dc1e 2023-04-08 22:04:59,927 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,927 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/cf2/e4f715ebb2e749ec90f7deb091ecb56d 2023-04-08 22:04:59,928 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/8a43fb69950c4167727f2a330ede3bf4/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,930 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf/recovered.edits/85.seqid 2023-04-08 22:04:59,930 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,931 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/7c2df3e0b95a4f8db65abff9b278100c to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf1/7c2df3e0b95a4f8db65abff9b278100c 2023-04-08 22:04:59,933 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/1e29fee01e3947ce818a79252648f47f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/cf2/1e29fee01e3947ce818a79252648f47f 2023-04-08 22:04:59,935 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4/recovered.edits/90.seqid 2023-04-08 22:04:59,935 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/8a43fb69950c4167727f2a330ede3bf4 2023-04-08 22:04:59,939 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/7f1b174f8196439bb4644eeb814fceb7.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,940 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/862142fe15004bb7a45aeab624443d3d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,941 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/9d803c121dfe4cec88e084231f6d8585.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,942 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/a0627e7879334c8bab18933f8ca0dc1e.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,943 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/12baaa72406c0bdceee750f3da4ff69d/cf2/e4f715ebb2e749ec90f7deb091ecb56d.574e7361fc669d4c8cafbcd53e38dfcf 2023-04-08 22:04:59,945 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/4c98cc28340d485e8478c9f84b400042 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf1/4c98cc28340d485e8478c9f84b400042 2023-04-08 22:04:59,946 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/974d02517dc3478c8964b3d3a91e7f3a to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/cf2/974d02517dc3478c8964b3d3a91e7f3a 2023-04-08 22:04:59,948 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d/recovered.edits/90.seqid 2023-04-08 22:04:59,949 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testRecoveryAndDoubleExecution/12baaa72406c0bdceee750f3da4ff69d 2023-04-08 22:04:59,949 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-04-08 22:04:59,950 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,952 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-04-08 22:04:59,953 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-04-08 22:04:59,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,954 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-04-08 22:04:59,954 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991499954"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:59,954 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991499954"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:59,954 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991499954"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:59,955 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:04:59,955 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 574e7361fc669d4c8cafbcd53e38dfcf, NAME => 'testRecoveryAndDoubleExecution,,1680991493823.574e7361fc669d4c8cafbcd53e38dfcf.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 8a43fb69950c4167727f2a330ede3bf4, NAME => 'testRecoveryAndDoubleExecution,,1680991495716.8a43fb69950c4167727f2a330ede3bf4.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 12baaa72406c0bdceee750f3da4ff69d, NAME => 'testRecoveryAndDoubleExecution,41,1680991495716.12baaa72406c0bdceee750f3da4ff69d.', STARTKEY => '41', ENDKEY => ''}] 2023-04-08 22:04:59,955 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-04-08 22:04:59,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991499956"}]},"ts":"9223372036854775807"} 2023-04-08 22:04:59,957 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-04-08 22:04:59,958 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-04-08 22:04:59,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 73 msec 2023-04-08 22:04:59,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-04-08 22:04:59,993 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-04-08 22:05:00,009 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 368) - Thread LEAK? -, OpenFileDescriptor=833 (was 737) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=103 (was 86) - SystemLoadAverage LEAK? -, ProcessCount=163 (was 163), AvailableMemoryMB=4839 (was 4871) 2023-04-08 22:05:00,024 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=103, ProcessCount=163, AvailableMemoryMB=4838 2023-04-08 22:05:00,025 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:05:00,028 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:05:00,028 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:05:00,029 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,031 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:05:00,032 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:05:00,034 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,034 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da empty. 2023-04-08 22:05:00,035 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,035 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-04-08 22:05:00,047 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-04-08 22:05:00,049 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => e5adb1828ae5eb94d8c554764a2e17da, NAME => 'testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing e5adb1828ae5eb94d8c554764a2e17da, disabling compactions & flushes 2023-04-08 22:05:00,059 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. after waiting 0 ms 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,059 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,059 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:00,062 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:05:00,063 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991500063"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991500063"}]},"ts":"1680991500063"} 2023-04-08 22:05:00,065 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:05:00,066 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:05:00,066 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991500066"}]},"ts":"1680991500066"} 2023-04-08 22:05:00,067 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-04-08 22:05:00,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:05:00,071 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:05:00,071 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:05:00,071 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:05:00,071 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:05:00,071 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, ASSIGN}] 2023-04-08 22:05:00,073 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, ASSIGN 2023-04-08 22:05:00,073 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:05:00,223 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:05:00,226 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=e5adb1828ae5eb94d8c554764a2e17da, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,226 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991500226"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991500226"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991500226"}]},"ts":"1680991500226"} 2023-04-08 22:05:00,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure e5adb1828ae5eb94d8c554764a2e17da, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:00,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e5adb1828ae5eb94d8c554764a2e17da, NAME => 'testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:05:00,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:00,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:00,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,394 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,396 DEBUG [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1 2023-04-08 22:05:00,396 DEBUG [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1 2023-04-08 22:05:00,396 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5adb1828ae5eb94d8c554764a2e17da columnFamilyName cf1 2023-04-08 22:05:00,397 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] regionserver.HStore(310): Store=e5adb1828ae5eb94d8c554764a2e17da/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:00,397 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,398 DEBUG [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2 2023-04-08 22:05:00,398 DEBUG [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2 2023-04-08 22:05:00,399 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e5adb1828ae5eb94d8c554764a2e17da columnFamilyName cf2 2023-04-08 22:05:00,399 INFO [StoreOpener-e5adb1828ae5eb94d8c554764a2e17da-1] regionserver.HStore(310): Store=e5adb1828ae5eb94d8c554764a2e17da/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:00,400 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,403 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:00,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:00,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:05:00,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened e5adb1828ae5eb94d8c554764a2e17da; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69800752, jitterRate=0.04011225700378418}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:00,410 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:00,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da., pid=127, masterSystemTime=1680991500384 2023-04-08 22:05:00,413 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,413 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=e5adb1828ae5eb94d8c554764a2e17da, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,414 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991500413"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991500413"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991500413"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991500413"}]},"ts":"1680991500413"} 2023-04-08 22:05:00,417 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-04-08 22:05:00,418 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure e5adb1828ae5eb94d8c554764a2e17da, server=jenkins-hbase20.apache.org,41749,1680991397967 in 185 msec 2023-04-08 22:05:00,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-04-08 22:05:00,420 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, ASSIGN in 347 msec 2023-04-08 22:05:00,420 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:05:00,421 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991500421"}]},"ts":"1680991500421"} 2023-04-08 22:05:00,422 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-04-08 22:05:00,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:05:00,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 396 msec 2023-04-08 22:05:00,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:00,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,545 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,545 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,545 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:00,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:00,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,546 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,547 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,547 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,547 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,547 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,547 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,548 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,548 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,548 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,550 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,550 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,550 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,551 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,551 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:00,551 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:00,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,552 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,553 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,553 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:05:00,553 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,553 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,554 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,554 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,554 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,566 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/d507c56493244c3fb2f02960dcd30746 2023-04-08 22:05:00,579 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/489bed7b1402409dad2f993de8cbc6ef 2023-04-08 22:05:00,583 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/d507c56493244c3fb2f02960dcd30746 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746 2023-04-08 22:05:00,588 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:05:00,589 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/489bed7b1402409dad2f993de8cbc6ef as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef 2023-04-08 22:05:00,594 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:05:00,595 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 42ms, sequenceid=6, compaction requested=false 2023-04-08 22:05:00,595 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,596 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:00,596 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,596 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:00,596 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:00,596 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,596 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,596 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,598 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,598 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,598 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,598 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:00,601 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c8a90c[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:00,601 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:00,601 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,601 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,601 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,601 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,601 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,602 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,602 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:00,602 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:00,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:00,602 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,603 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,603 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:00,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:00,606 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:00,606 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:00,606 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:00,606 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,606 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:00,606 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,606 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,606 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,606 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,607 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,607 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,607 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,608 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,609 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,609 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,609 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,612 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,612 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:00,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:00,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,612 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,612 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,613 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,613 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,654 ERROR [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1236918431-148.251.75.209-1680991393780 (Datanode Uuid 77a94139-75e1-486c-a582-be7e9aa2826d) service to localhost.localdomain/127.0.0.1:34893 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-08 22:05:00,713 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:00,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:00,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:00,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:00,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,738 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,738 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,738 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:00,738 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:00,738 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,739 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,740 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,740 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,740 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,742 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:00,743 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,743 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:00,743 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,744 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:00,744 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,745 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,745 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,755 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/2634a4200677432bab9834f53902cfb3 2023-04-08 22:05:00,771 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f 2023-04-08 22:05:00,776 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/2634a4200677432bab9834f53902cfb3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3 2023-04-08 22:05:00,781 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:05:00,782 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f 2023-04-08 22:05:00,787 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:05:00,788 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 44ms, sequenceid=19, compaction requested=false 2023-04-08 22:05:00,788 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:00,788 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,788 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:00,788 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:00,788 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,789 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,789 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,790 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,790 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,790 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,790 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:00,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a6731ad[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:00,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:00,793 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,793 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,793 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,793 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,794 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,794 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:00,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,794 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,795 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,795 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,796 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,797 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,798 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:00,799 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:00,799 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:00,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:00,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:00,799 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,799 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,801 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,800 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,800 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,801 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,801 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,802 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,804 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,805 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,805 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,811 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,811 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,811 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,812 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,811 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,812 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:00,813 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,812 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,813 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,913 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:00,915 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:00,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:00,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:00,938 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,938 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:00,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:00,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,939 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,939 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,939 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,939 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,941 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,941 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:00,941 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,941 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,941 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,941 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:00,942 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,943 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,943 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,943 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,943 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,944 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,953 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/8fad5591256049f49824fb3797a47e10 2023-04-08 22:05:00,964 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/306cab03cf724e909461eb895991c567 2023-04-08 22:05:00,967 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/8fad5591256049f49824fb3797a47e10 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10 2023-04-08 22:05:00,969 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:05:00,970 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/306cab03cf724e909461eb895991c567 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567 2023-04-08 22:05:00,973 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:05:00,973 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 32ms, sequenceid=32, compaction requested=true 2023-04-08 22:05:00,974 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:00,974 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:00,974 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:00,974 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:00,974 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,974 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:00,974 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:00,975 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,975 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:00,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:00,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45ed315b[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:00,977 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:00,977 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,977 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,977 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,978 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:00,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:00,978 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,979 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,979 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,979 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:00,981 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:00,981 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:00,981 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:00,981 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:00,981 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,981 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,982 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:00,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:00,982 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,983 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,983 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:00,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,984 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,984 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:00,984 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:00,986 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:00,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:00,986 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:00,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,986 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:00,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,987 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:00,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:00,987 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,087 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:01,088 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:01,107 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:01,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:01,109 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,109 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:01,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:01,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,110 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,110 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,111 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,111 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,110 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,111 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,111 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:01,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,112 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,113 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,113 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,113 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,113 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:01,114 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,114 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:01,114 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:01,114 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,115 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,115 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,126 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/ec10abbe913e45558b0c6a37281000bc 2023-04-08 22:05:01,136 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/542d7ebdef9c4a1380a6791f44190072 2023-04-08 22:05:01,140 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/ec10abbe913e45558b0c6a37281000bc as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc 2023-04-08 22:05:01,143 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:05:01,144 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/542d7ebdef9c4a1380a6791f44190072 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072 2023-04-08 22:05:01,148 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:05:01,149 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 35ms, sequenceid=45, compaction requested=true 2023-04-08 22:05:01,149 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:01,149 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,149 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:01,149 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:01,149 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,149 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,149 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,150 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,150 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,151 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:01,153 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51b629b[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:01,153 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:01,153 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,153 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,153 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:01,154 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,154 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:01,155 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,155 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,156 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,156 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:01,159 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:01,159 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:01,159 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:01,159 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:01,159 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,159 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,160 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,161 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,161 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,161 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,161 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,162 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,162 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,164 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,165 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,165 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,173 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,173 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,173 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,174 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:01,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,173 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,174 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:01,174 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:01,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,174 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,174 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,174 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,175 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,275 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:01,277 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:01,296 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:01,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:01,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,298 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,298 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,298 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:01,298 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:01,299 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,299 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,300 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,300 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,300 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,300 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,301 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,301 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:01,300 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,302 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,302 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,303 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,303 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,303 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:01,303 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:01,303 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,304 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,304 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,305 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,305 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,305 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,315 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/4144166cc1274413936c84db74d45f74 2023-04-08 22:05:01,331 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/19ec976882404fa6bf9edf01c34a8cd4 2023-04-08 22:05:01,338 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/4144166cc1274413936c84db74d45f74 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74 2023-04-08 22:05:01,344 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:05:01,345 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/19ec976882404fa6bf9edf01c34a8cd4 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4 2023-04-08 22:05:01,350 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:05:01,351 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 46ms, sequenceid=58, compaction requested=true 2023-04-08 22:05:01,351 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:01,351 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,351 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:01,351 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:01,351 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,351 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,351 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,352 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,352 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,353 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,353 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:01,357 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@425c3bfd[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:01,357 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:01,357 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,358 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,358 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,358 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:01,358 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:01,358 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,359 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,359 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,359 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:01,359 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,359 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,360 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,360 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,361 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,361 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:01,367 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:01,367 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:01,367 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:01,367 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:01,367 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,367 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,368 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,369 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,370 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,369 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,370 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,372 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,373 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,373 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,378 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,379 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter because node does not exist (not an error) 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,379 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,378 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,379 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,379 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:01,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,378 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,378 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,380 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,379 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:01,379 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,380 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,380 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,380 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,480 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:01,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:01,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:01,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:01,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,507 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,507 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,507 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-04-08 22:05:01,507 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-04-08 22:05:01,507 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,507 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,508 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,508 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,509 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,509 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:01,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,511 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,511 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:01,512 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:01,512 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,512 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. started... 2023-04-08 22:05:01,512 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,514 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,525 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/a90086547a8b468d9fc3cb0d8bb1b155 2023-04-08 22:05:01,536 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/d238319dc5c6450eb4574ce3501bddee 2023-04-08 22:05:01,540 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/a90086547a8b468d9fc3cb0d8bb1b155 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155 2023-04-08 22:05:01,542 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:05:01,543 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/d238319dc5c6450eb4574ce3501bddee as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee 2023-04-08 22:05:01,546 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:05:01,546 INFO [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 34ms, sequenceid=71, compaction requested=true 2023-04-08 22:05:01,546 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:01,546 DEBUG [rs(jenkins-hbase20.apache.org,41749,1680991397967)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,547 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:01,547 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:01,547 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,547 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-04-08 22:05:01,547 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-04-08 22:05:01,548 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,548 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,548 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,548 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:01,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-04-08 22:05:01,550 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31458042[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:01,550 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-04-08 22:05:01,550 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,551 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,551 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-04-08 22:05:01,551 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,552 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,553 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,552 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,553 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-04-08 22:05:01,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,553 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,554 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,555 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase20.apache.org,41749,1680991397967': 2023-04-08 22:05:01,557 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-04-08 22:05:01,557 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-04-08 22:05:01,557 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:01,557 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:01,557 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,557 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,558 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:01,558 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,559 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,559 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,559 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,559 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,561 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:01,561 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,561 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,563 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:01,563 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-04-08 22:05:01,564 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:01,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:01,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:01,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:01,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,564 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-04-08 22:05:01,564 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,565 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:01,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:01,664 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:01,665 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:01,674 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,675 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e5adb1828ae5eb94d8c554764a2e17da, daughterA=c10f6256a1924f9a57c1fdf60eda6331, daughterB=2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:01,676 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e5adb1828ae5eb94d8c554764a2e17da, daughterA=c10f6256a1924f9a57c1fdf60eda6331, daughterB=2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:01,676 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e5adb1828ae5eb94d8c554764a2e17da, daughterA=c10f6256a1924f9a57c1fdf60eda6331, daughterB=2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:01,676 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e5adb1828ae5eb94d8c554764a2e17da, daughterA=c10f6256a1924f9a57c1fdf60eda6331, daughterB=2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:01,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, UNASSIGN}] 2023-04-08 22:05:01,679 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, UNASSIGN 2023-04-08 22:05:01,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=e5adb1828ae5eb94d8c554764a2e17da, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:01,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991501680"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991501680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991501680"}]},"ts":"1680991501680"} 2023-04-08 22:05:01,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure e5adb1828ae5eb94d8c554764a2e17da, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:01,834 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,834 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing e5adb1828ae5eb94d8c554764a2e17da, disabling compactions & flushes 2023-04-08 22:05:01,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. after waiting 0 ms 2023-04-08 22:05:01,835 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing e5adb1828ae5eb94d8c554764a2e17da 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:05:01,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/f5200c711b8447469cd7dca5887d87f3 2023-04-08 22:05:01,862 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/aafa37ea179741649cf10998b19caa66 2023-04-08 22:05:01,866 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf1/f5200c711b8447469cd7dca5887d87f3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3 2023-04-08 22:05:01,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:05:01,873 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.tmp/cf2/aafa37ea179741649cf10998b19caa66 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66 2023-04-08 22:05:01,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:05:01,880 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e5adb1828ae5eb94d8c554764a2e17da in 45ms, sequenceid=82, compaction requested=true 2023-04-08 22:05:01,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:05:01,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:01,886 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. 2023-04-08 22:05:01,886 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for e5adb1828ae5eb94d8c554764a2e17da: 2023-04-08 22:05:01,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=e5adb1828ae5eb94d8c554764a2e17da, regionState=CLOSED 2023-04-08 22:05:01,887 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991501887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991501887"}]},"ts":"1680991501887"} 2023-04-08 22:05:01,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-04-08 22:05:01,890 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure e5adb1828ae5eb94d8c554764a2e17da, server=jenkins-hbase20.apache.org,41749,1680991397967 in 207 msec 2023-04-08 22:05:01,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-04-08 22:05:01,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e5adb1828ae5eb94d8c554764a2e17da, UNASSIGN in 212 msec 2023-04-08 22:05:01,901 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=e5adb1828ae5eb94d8c554764a2e17da, threads=14 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,901 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,902 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,902 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,902 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,902 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,906 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,908 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072, top=false 2023-04-08 22:05:01,915 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567, top=false 2023-04-08 22:05:01,915 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66, top=false 2023-04-08 22:05:01,915 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f, top=false 2023-04-08 22:05:01,916 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3, top=false 2023-04-08 22:05:01,916 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4, top=false 2023-04-08 22:05:01,917 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3, top=false 2023-04-08 22:05:01,918 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee, top=false 2023-04-08 22:05:01,918 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef, top=false 2023-04-08 22:05:01,919 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc, top=false 2023-04-08 22:05:01,920 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746, top=false 2023-04-08 22:05:01,920 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155, top=false 2023-04-08 22:05:01,923 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74, top=false 2023-04-08 22:05:01,924 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10, top=false 2023-04-08 22:05:01,926 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,931 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,938 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,938 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,938 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,938 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,941 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,941 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,941 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,941 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,942 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,944 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,950 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,950 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,950 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,950 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,950 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,951 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,951 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10 for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,951 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/.splits/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee for child: c10f6256a1924f9a57c1fdf60eda6331, parent: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,955 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,956 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,956 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,956 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,956 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,957 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,958 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,958 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10 for region: e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:01,958 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region e5adb1828ae5eb94d8c554764a2e17da Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-04-08 22:05:01,978 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:05:01,979 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:05:01,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1680991501980"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1680991501980"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1680991501980"}]},"ts":"1680991501980"} 2023-04-08 22:05:01,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991501980"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991501980"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991501980"}]},"ts":"1680991501980"} 2023-04-08 22:05:01,981 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991501980"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991501980"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991501980"}]},"ts":"1680991501980"} 2023-04-08 22:05:01,983 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, ASSIGN}] 2023-04-08 22:05:01,984 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, ASSIGN 2023-04-08 22:05:01,984 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:05:01,985 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, ASSIGN 2023-04-08 22:05:01,985 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41749,1680991397967; forceNewPlan=false, retain=false 2023-04-08 22:05:02,135 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:05:02,136 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=c10f6256a1924f9a57c1fdf60eda6331, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,136 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502136"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991502136"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991502136"}]},"ts":"1680991502136"} 2023-04-08 22:05:02,137 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure c10f6256a1924f9a57c1fdf60eda6331, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:02,140 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=2ef8b5a056c944a6a97c0afa793925b4, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502140"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991502140"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991502140"}]},"ts":"1680991502140"} 2023-04-08 22:05:02,141 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 2ef8b5a056c944a6a97c0afa793925b4, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:02,241 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:05:02,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2ef8b5a056c944a6a97c0afa793925b4, NAME => 'testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.', STARTKEY => '71', ENDKEY => ''} 2023-04-08 22:05:02,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:02,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:02,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,293 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,294 DEBUG [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf1 2023-04-08 22:05:02,294 DEBUG [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf1 2023-04-08 22:05:02,294 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ef8b5a056c944a6a97c0afa793925b4 columnFamilyName cf1 2023-04-08 22:05:02,295 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] regionserver.HStore(310): Store=2ef8b5a056c944a6a97c0afa793925b4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:02,295 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,296 DEBUG [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf2 2023-04-08 22:05:02,296 DEBUG [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf2 2023-04-08 22:05:02,297 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2ef8b5a056c944a6a97c0afa793925b4 columnFamilyName cf2 2023-04-08 22:05:02,297 INFO [StoreOpener-2ef8b5a056c944a6a97c0afa793925b4-1] regionserver.HStore(310): Store=2ef8b5a056c944a6a97c0afa793925b4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:02,298 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,301 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:02,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 2ef8b5a056c944a6a97c0afa793925b4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67261845, jitterRate=0.0022795945405960083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:02,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 2ef8b5a056c944a6a97c0afa793925b4: 2023-04-08 22:05:02,303 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4., pid=134, masterSystemTime=1680991502289 2023-04-08 22:05:02,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c10f6256a1924f9a57c1fdf60eda6331, NAME => 'testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.', STARTKEY => '', ENDKEY => '71'} 2023-04-08 22:05:02,305 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=2ef8b5a056c944a6a97c0afa793925b4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,305 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502305"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991502305"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991502305"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991502305"}]},"ts":"1680991502305"} 2023-04-08 22:05:02,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:02,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:02,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,307 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,308 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1 2023-04-08 22:05:02,308 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1 2023-04-08 22:05:02,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-04-08 22:05:02,309 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c10f6256a1924f9a57c1fdf60eda6331 columnFamilyName cf1 2023-04-08 22:05:02,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 2ef8b5a056c944a6a97c0afa793925b4, server=jenkins-hbase20.apache.org,41749,1680991397967 in 166 msec 2023-04-08 22:05:02,310 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, ASSIGN in 326 msec 2023-04-08 22:05:02,315 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3 2023-04-08 22:05:02,318 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74 2023-04-08 22:05:02,320 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10 2023-04-08 22:05:02,323 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155 2023-04-08 22:05:02,325 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746 2023-04-08 22:05:02,328 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc 2023-04-08 22:05:02,330 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3 2023-04-08 22:05:02,330 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(310): Store=c10f6256a1924f9a57c1fdf60eda6331/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:02,330 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,331 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2 2023-04-08 22:05:02,331 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2 2023-04-08 22:05:02,332 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c10f6256a1924f9a57c1fdf60eda6331 columnFamilyName cf2 2023-04-08 22:05:02,336 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4 2023-04-08 22:05:02,339 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567 2023-04-08 22:05:02,341 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef 2023-04-08 22:05:02,344 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072 2023-04-08 22:05:02,347 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66 2023-04-08 22:05:02,351 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee 2023-04-08 22:05:02,354 DEBUG [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f 2023-04-08 22:05:02,354 INFO [StoreOpener-c10f6256a1924f9a57c1fdf60eda6331-1] regionserver.HStore(310): Store=c10f6256a1924f9a57c1fdf60eda6331/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:02,355 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:02,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,358 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened c10f6256a1924f9a57c1fdf60eda6331; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69375410, jitterRate=0.03377416729927063}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:02,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for c10f6256a1924f9a57c1fdf60eda6331: 2023-04-08 22:05:02,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331., pid=133, masterSystemTime=1680991502289 2023-04-08 22:05:02,359 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:05:02,360 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1912): c10f6256a1924f9a57c1fdf60eda6331/cf2 is initiating minor compaction (all files) 2023-04-08 22:05:02,360 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1912): c10f6256a1924f9a57c1fdf60eda6331/cf1 is initiating minor compaction (all files) 2023-04-08 22:05:02,360 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c10f6256a1924f9a57c1fdf60eda6331/cf2 in testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,360 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c10f6256a1924f9a57c1fdf60eda6331/cf1 in testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,361 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/.tmp, totalSize=34.9 K 2023-04-08 22:05:02,361 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/.tmp, totalSize=34.9 K 2023-04-08 22:05:02,361 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991500533 2023-04-08 22:05:02,361 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991500533 2023-04-08 22:05:02,361 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,361 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991500716 2023-04-08 22:05:02,362 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,361 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991500716 2023-04-08 22:05:02,362 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991500917 2023-04-08 22:05:02,362 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=c10f6256a1924f9a57c1fdf60eda6331, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,362 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502362"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991502362"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991502362"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991502362"}]},"ts":"1680991502362"} 2023-04-08 22:05:02,362 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991500917 2023-04-08 22:05:02,362 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991501089 2023-04-08 22:05:02,362 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991501089 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991501279 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991501279 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991501485 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991501485 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991501666 2023-04-08 22:05:02,363 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991501666 2023-04-08 22:05:02,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-04-08 22:05:02,365 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure c10f6256a1924f9a57c1fdf60eda6331, server=jenkins-hbase20.apache.org,41749,1680991397967 in 226 msec 2023-04-08 22:05:02,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-04-08 22:05:02,366 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, ASSIGN in 382 msec 2023-04-08 22:05:02,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=e5adb1828ae5eb94d8c554764a2e17da, daughterA=c10f6256a1924f9a57c1fdf60eda6331, daughterB=2ef8b5a056c944a6a97c0afa793925b4 in 693 msec 2023-04-08 22:05:02,379 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] throttle.PressureAwareThroughputController(145): c10f6256a1924f9a57c1fdf60eda6331#cf2#compaction#124 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:02,380 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] throttle.PressureAwareThroughputController(145): c10f6256a1924f9a57c1fdf60eda6331#cf1#compaction#125 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:02,395 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/.tmp/cf1/f701527398164a509f343e08b1c1e9db as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/f701527398164a509f343e08b1c1e9db 2023-04-08 22:05:02,395 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/.tmp/cf2/1ed2776f58f741b983f6b3e08c248533 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/1ed2776f58f741b983f6b3e08c248533 2023-04-08 22:05:02,398 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c10f6256a1924f9a57c1fdf60eda6331/cf1 of c10f6256a1924f9a57c1fdf60eda6331 into f701527398164a509f343e08b1c1e9db(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:02,398 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c10f6256a1924f9a57c1fdf60eda6331/cf2 of c10f6256a1924f9a57c1fdf60eda6331 into 1ed2776f58f741b983f6b3e08c248533(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:02,398 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c10f6256a1924f9a57c1fdf60eda6331: 2023-04-08 22:05:02,398 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c10f6256a1924f9a57c1fdf60eda6331: 2023-04-08 22:05:02,398 INFO [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331., storeName=c10f6256a1924f9a57c1fdf60eda6331/cf1, priority=9, startTime=1680991502359; duration=0sec 2023-04-08 22:05:02,398 INFO [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331., storeName=c10f6256a1924f9a57c1fdf60eda6331/cf2, priority=9, startTime=1680991502359; duration=0sec 2023-04-08 22:05:02,399 DEBUG [RS:1;jenkins-hbase20:41749-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:02,399 DEBUG [RS:1;jenkins-hbase20:41749-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:02,426 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41749] ipc.CallRunner(144): callId: 1019 service: ClientService methodName: Scan size: 157 connection: 148.251.75.209:38532 deadline: 1680991562426, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da. is not online on jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,537 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-04-08 22:05:02,541 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:05:02,543 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,543 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,547 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-08 22:05:02,547 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991502547"}]},"ts":"1680991502547"} 2023-04-08 22:05:02,548 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-04-08 22:05:02,549 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-04-08 22:05:02,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, UNASSIGN}] 2023-04-08 22:05:02,551 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, UNASSIGN 2023-04-08 22:05:02,552 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=2ef8b5a056c944a6a97c0afa793925b4, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,552 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502552"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991502552"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991502552"}]},"ts":"1680991502552"} 2023-04-08 22:05:02,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 2ef8b5a056c944a6a97c0afa793925b4, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:02,556 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, UNASSIGN 2023-04-08 22:05:02,557 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=c10f6256a1924f9a57c1fdf60eda6331, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:02,557 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502557"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991502557"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991502557"}]},"ts":"1680991502557"} 2023-04-08 22:05:02,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure c10f6256a1924f9a57c1fdf60eda6331, server=jenkins-hbase20.apache.org,41749,1680991397967}] 2023-04-08 22:05:02,648 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-08 22:05:02,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing c10f6256a1924f9a57c1fdf60eda6331, disabling compactions & flushes 2023-04-08 22:05:02,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. after waiting 0 ms 2023-04-08 22:05:02,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,708 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3] to archive 2023-04-08 22:05:02,709 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:02,713 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d507c56493244c3fb2f02960dcd30746 2023-04-08 22:05:02,716 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-2634a4200677432bab9834f53902cfb3 2023-04-08 22:05:02,719 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-8fad5591256049f49824fb3797a47e10 2023-04-08 22:05:02,721 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-ec10abbe913e45558b0c6a37281000bc 2023-04-08 22:05:02,723 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-4144166cc1274413936c84db74d45f74 2023-04-08 22:05:02,726 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-a90086547a8b468d9fc3cb0d8bb1b155 2023-04-08 22:05:02,728 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-f5200c711b8447469cd7dca5887d87f3 2023-04-08 22:05:02,730 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66] to archive 2023-04-08 22:05:02,731 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:02,733 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-489bed7b1402409dad2f993de8cbc6ef 2023-04-08 22:05:02,735 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-e4ef5bc96cc94dcdbf80861e67c9d81f 2023-04-08 22:05:02,737 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-306cab03cf724e909461eb895991c567 2023-04-08 22:05:02,739 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-542d7ebdef9c4a1380a6791f44190072 2023-04-08 22:05:02,741 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-19ec976882404fa6bf9edf01c34a8cd4 2023-04-08 22:05:02,743 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-d238319dc5c6450eb4574ce3501bddee 2023-04-08 22:05:02,745 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/testSplitTableRegionEmptyDaughter=e5adb1828ae5eb94d8c554764a2e17da-aafa37ea179741649cf10998b19caa66 2023-04-08 22:05:02,750 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:05:02,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:02,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331. 2023-04-08 22:05:02,751 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for c10f6256a1924f9a57c1fdf60eda6331: 2023-04-08 22:05:02,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,753 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 2ef8b5a056c944a6a97c0afa793925b4, disabling compactions & flushes 2023-04-08 22:05:02,754 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,754 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=c10f6256a1924f9a57c1fdf60eda6331, regionState=CLOSED 2023-04-08 22:05:02,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. after waiting 0 ms 2023-04-08 22:05:02,754 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502754"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991502754"}]},"ts":"1680991502754"} 2023-04-08 22:05:02,754 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-04-08 22:05:02,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure c10f6256a1924f9a57c1fdf60eda6331, server=jenkins-hbase20.apache.org,41749,1680991397967 in 197 msec 2023-04-08 22:05:02,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-04-08 22:05:02,758 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:02,759 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4. 2023-04-08 22:05:02,759 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 2ef8b5a056c944a6a97c0afa793925b4: 2023-04-08 22:05:02,759 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=c10f6256a1924f9a57c1fdf60eda6331, UNASSIGN in 207 msec 2023-04-08 22:05:02,760 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,760 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=2ef8b5a056c944a6a97c0afa793925b4, regionState=CLOSED 2023-04-08 22:05:02,761 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1680991502760"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991502760"}]},"ts":"1680991502760"} 2023-04-08 22:05:02,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-04-08 22:05:02,763 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 2ef8b5a056c944a6a97c0afa793925b4, server=jenkins-hbase20.apache.org,41749,1680991397967 in 208 msec 2023-04-08 22:05:02,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-04-08 22:05:02,764 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=2ef8b5a056c944a6a97c0afa793925b4, UNASSIGN in 213 msec 2023-04-08 22:05:02,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991502764"}]},"ts":"1680991502764"} 2023-04-08 22:05:02,765 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-04-08 22:05:02,766 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-04-08 22:05:02,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 223 msec 2023-04-08 22:05:02,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-04-08 22:05:02,851 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-04-08 22:05:02,853 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,854 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,858 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,859 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-04-08 22:05:02,864 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:02,864 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,864 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,866 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/recovered.edits] 2023-04-08 22:05:02,866 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/recovered.edits] 2023-04-08 22:05:02,866 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/recovered.edits] 2023-04-08 22:05:02,870 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/f701527398164a509f343e08b1c1e9db to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf1/f701527398164a509f343e08b1c1e9db 2023-04-08 22:05:02,870 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/2634a4200677432bab9834f53902cfb3 2023-04-08 22:05:02,872 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/4144166cc1274413936c84db74d45f74 2023-04-08 22:05:02,872 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/1ed2776f58f741b983f6b3e08c248533 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/cf2/1ed2776f58f741b983f6b3e08c248533 2023-04-08 22:05:02,872 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/recovered.edits/88.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4/recovered.edits/88.seqid 2023-04-08 22:05:02,873 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/8fad5591256049f49824fb3797a47e10 2023-04-08 22:05:02,873 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/2ef8b5a056c944a6a97c0afa793925b4 2023-04-08 22:05:02,874 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/a90086547a8b468d9fc3cb0d8bb1b155 2023-04-08 22:05:02,874 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331/recovered.edits/90.seqid 2023-04-08 22:05:02,874 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/d507c56493244c3fb2f02960dcd30746 2023-04-08 22:05:02,875 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/c10f6256a1924f9a57c1fdf60eda6331 2023-04-08 22:05:02,875 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/ec10abbe913e45558b0c6a37281000bc 2023-04-08 22:05:02,876 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf1/f5200c711b8447469cd7dca5887d87f3 2023-04-08 22:05:02,878 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/19ec976882404fa6bf9edf01c34a8cd4 2023-04-08 22:05:02,879 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/306cab03cf724e909461eb895991c567 2023-04-08 22:05:02,880 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/489bed7b1402409dad2f993de8cbc6ef 2023-04-08 22:05:02,881 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/542d7ebdef9c4a1380a6791f44190072 2023-04-08 22:05:02,882 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/aafa37ea179741649cf10998b19caa66 2023-04-08 22:05:02,882 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/d238319dc5c6450eb4574ce3501bddee 2023-04-08 22:05:02,883 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/cf2/e4ef5bc96cc94dcdbf80861e67c9d81f 2023-04-08 22:05:02,885 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da/recovered.edits/85.seqid 2023-04-08 22:05:02,886 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegionEmptyDaughter/e5adb1828ae5eb94d8c554764a2e17da 2023-04-08 22:05:02,886 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-04-08 22:05:02,888 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,889 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-04-08 22:05:02,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-04-08 22:05:02,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-04-08 22:05:02,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991502893"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:02,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991502893"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:02,893 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991502893"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:02,895 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:05:02,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e5adb1828ae5eb94d8c554764a2e17da, NAME => 'testSplitTableRegionEmptyDaughter,,1680991500028.e5adb1828ae5eb94d8c554764a2e17da.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c10f6256a1924f9a57c1fdf60eda6331, NAME => 'testSplitTableRegionEmptyDaughter,,1680991501674.c10f6256a1924f9a57c1fdf60eda6331.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 2ef8b5a056c944a6a97c0afa793925b4, NAME => 'testSplitTableRegionEmptyDaughter,71,1680991501674.2ef8b5a056c944a6a97c0afa793925b4.', STARTKEY => '71', ENDKEY => ''}] 2023-04-08 22:05:02,895 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-04-08 22:05:02,895 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991502895"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:02,896 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-04-08 22:05:02,898 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-04-08 22:05:02,898 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 45 msec 2023-04-08 22:05:02,961 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-04-08 22:05:02,961 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-04-08 22:05:02,980 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 377), OpenFileDescriptor=833 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 103), ProcessCount=162 (was 163), AvailableMemoryMB=4834 (was 4838) 2023-04-08 22:05:02,994 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=833, MaxFileDescriptor=60000, SystemLoadAverage=95, ProcessCount=162, AvailableMemoryMB=4834 2023-04-08 22:05:02,994 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:05:02,997 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-04-08 22:05:02,997 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(492): Client=jenkins//148.251.75.209 set balanceSwitch=false 2023-04-08 22:05:02,998 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-04-08 22:05:03,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-04-08 22:05:03,000 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-04-08 22:05:03,001 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,002 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f empty. 2023-04-08 22:05:03,002 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,002 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-04-08 22:05:03,012 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-04-08 22:05:03,013 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 75da9be8f486bf115207274ff1255b9f, NAME => 'testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 75da9be8f486bf115207274ff1255b9f, disabling compactions & flushes 2023-04-08 22:05:03,023 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. after waiting 0 ms 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,023 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,023 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:03,026 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-04-08 22:05:03,027 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991503027"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991503027"}]},"ts":"1680991503027"} 2023-04-08 22:05:03,028 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-04-08 22:05:03,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-04-08 22:05:03,029 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991503029"}]},"ts":"1680991503029"} 2023-04-08 22:05:03,031 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-04-08 22:05:03,034 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase20.apache.org=0} racks are {/default-rack=0} 2023-04-08 22:05:03,035 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-04-08 22:05:03,035 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-04-08 22:05:03,035 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-04-08 22:05:03,035 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-04-08 22:05:03,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, ASSIGN}] 2023-04-08 22:05:03,037 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, ASSIGN 2023-04-08 22:05:03,038 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, ASSIGN; state=OFFLINE, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:05:03,189 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-04-08 22:05:03,191 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=75da9be8f486bf115207274ff1255b9f, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,191 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991503191"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991503191"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991503191"}]},"ts":"1680991503191"} 2023-04-08 22:05:03,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 75da9be8f486bf115207274ff1255b9f, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:03,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,356 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75da9be8f486bf115207274ff1255b9f, NAME => 'testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.', STARTKEY => '', ENDKEY => ''} 2023-04-08 22:05:03,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:03,357 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:03,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,358 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,360 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,362 DEBUG [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1 2023-04-08 22:05:03,362 DEBUG [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1 2023-04-08 22:05:03,363 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75da9be8f486bf115207274ff1255b9f columnFamilyName cf1 2023-04-08 22:05:03,363 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] regionserver.HStore(310): Store=75da9be8f486bf115207274ff1255b9f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:03,363 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,364 DEBUG [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2 2023-04-08 22:05:03,364 DEBUG [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2 2023-04-08 22:05:03,364 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75da9be8f486bf115207274ff1255b9f columnFamilyName cf2 2023-04-08 22:05:03,365 INFO [StoreOpener-75da9be8f486bf115207274ff1255b9f-1] regionserver.HStore(310): Store=75da9be8f486bf115207274ff1255b9f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:03,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,365 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:03,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:03,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-04-08 22:05:03,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 75da9be8f486bf115207274ff1255b9f; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69081481, jitterRate=0.029394283890724182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:03,369 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:03,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f., pid=143, masterSystemTime=1680991503348 2023-04-08 22:05:03,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=75da9be8f486bf115207274ff1255b9f, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991503371"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991503371"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991503371"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991503371"}]},"ts":"1680991503371"} 2023-04-08 22:05:03,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-04-08 22:05:03,373 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 75da9be8f486bf115207274ff1255b9f, server=jenkins-hbase20.apache.org,41779,1680991398002 in 177 msec 2023-04-08 22:05:03,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-04-08 22:05:03,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, ASSIGN in 338 msec 2023-04-08 22:05:03,376 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-04-08 22:05:03,376 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991503376"}]},"ts":"1680991503376"} 2023-04-08 22:05:03,377 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-04-08 22:05:03,378 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-04-08 22:05:03,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 380 msec 2023-04-08 22:05:03,515 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:03,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:03,517 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:03,517 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,517 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:03,517 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:03,518 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,518 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,519 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,519 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,519 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,519 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,520 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,520 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,521 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:03,521 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:03,521 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,521 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:03,522 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=120 B heapSize=928 B 2023-04-08 22:05:03,522 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,523 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,523 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,523 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,523 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,524 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,533 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/ec5498f0626a4c8b8e90d8f7aa046812 2023-04-08 22:05:03,548 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/6cacb1af20c94c30a0cb9e8eacbb769b 2023-04-08 22:05:03,551 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/ec5498f0626a4c8b8e90d8f7aa046812 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812 2023-04-08 22:05:03,553 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:05:03,554 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/6cacb1af20c94c30a0cb9e8eacbb769b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b 2023-04-08 22:05:03,556 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:05:03,557 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 35ms, sequenceid=6, compaction requested=false 2023-04-08 22:05:03,557 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-04-08 22:05:03,557 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:03,557 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,558 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:03,558 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:03,558 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,558 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,558 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,559 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,559 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,559 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,559 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:03,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1d0e18d8[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:03,560 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:03,560 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,562 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:03,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,562 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,563 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,563 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:03,565 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:03,565 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:03,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:03,565 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:03,566 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:03,566 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,566 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,567 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,567 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,568 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,569 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,570 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,572 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:03,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,572 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,572 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:03,573 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,573 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,573 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,574 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,674 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:03,676 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:03,696 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:03,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:03,697 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:03,697 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:03,697 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,697 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:03,697 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:03,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,698 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,699 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,699 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:03,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,699 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,700 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,702 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,702 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,702 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,702 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,703 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,703 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:03,703 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:03,703 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,703 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:03,703 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:03,703 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,704 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,704 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,704 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,704 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,704 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,720 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/fb65bdfde2324d7087e3a54f965ef58e 2023-04-08 22:05:03,730 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 2023-04-08 22:05:03,733 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/fb65bdfde2324d7087e3a54f965ef58e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e 2023-04-08 22:05:03,736 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:05:03,736 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 2023-04-08 22:05:03,740 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3, entries=10, sequenceid=19, filesize=5.0 K 2023-04-08 22:05:03,740 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 37ms, sequenceid=19, compaction requested=false 2023-04-08 22:05:03,740 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:03,740 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,741 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:03,741 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:03,741 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,741 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,741 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,742 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,742 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,742 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,742 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:03,744 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@435fbe58[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:03,744 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:03,744 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,745 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,745 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:03,745 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,745 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:03,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,746 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,747 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,747 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,749 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:03,750 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:03,750 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:03,750 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:03,750 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:03,750 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:03,750 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:03,751 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,751 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,751 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,751 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,752 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,752 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,752 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,753 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,753 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,755 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,755 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,756 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,759 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,759 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:03,760 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:03,759 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,759 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:03,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:03,760 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,761 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,761 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,761 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,761 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,761 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,761 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,861 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:03,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:03,881 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:03,883 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:03,883 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:03,883 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,883 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:03,883 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:03,884 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,884 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,886 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,886 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,886 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,886 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:03,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:03,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,888 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:03,889 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:03,889 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,890 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:03,890 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,890 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,890 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:03,890 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:03,890 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,891 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,891 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,891 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,892 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,892 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,902 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/056f010dd37c4bd196cc728e7e22f67d 2023-04-08 22:05:03,916 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/bdcff42970c542f589885bdfac76b8fa 2023-04-08 22:05:03,920 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/056f010dd37c4bd196cc728e7e22f67d as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d 2023-04-08 22:05:03,923 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:05:03,924 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/bdcff42970c542f589885bdfac76b8fa as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa 2023-04-08 22:05:03,926 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa, entries=10, sequenceid=32, filesize=5.0 K 2023-04-08 22:05:03,927 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 37ms, sequenceid=32, compaction requested=true 2023-04-08 22:05:03,927 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:03,927 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:03,927 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:03,927 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:03,927 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,927 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:03,927 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:03,928 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,928 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,928 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,929 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:03,930 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@607ff61d[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:03,930 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:03,930 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,930 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,930 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,930 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:03,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,931 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:03,931 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,932 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,932 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:03,936 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:03,936 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:03,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:03,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:03,936 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:03,936 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:03,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,937 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,937 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,938 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:03,939 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:03,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:03,940 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,941 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,941 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,943 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:03,944 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:03,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,944 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:03,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:03,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:03,945 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:03,946 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,045 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:04,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:04,069 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:04,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:04,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:04,071 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:04,071 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,071 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:04,071 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:04,071 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,071 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,072 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,072 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,072 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,072 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,072 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,073 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,073 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,073 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,074 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,074 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,075 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,074 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,075 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,075 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,075 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:04,075 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:04,076 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,076 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,076 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,077 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,086 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/7af2681fb86b47269cdbc00f13d5917b 2023-04-08 22:05:04,096 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/56e42bdc6c7845ad915268372feaa6b9 2023-04-08 22:05:04,099 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/7af2681fb86b47269cdbc00f13d5917b as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b 2023-04-08 22:05:04,101 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:05:04,102 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/56e42bdc6c7845ad915268372feaa6b9 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9 2023-04-08 22:05:04,105 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9, entries=10, sequenceid=45, filesize=5.0 K 2023-04-08 22:05:04,105 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 30ms, sequenceid=45, compaction requested=true 2023-04-08 22:05:04,105 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:04,105 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,105 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:04,105 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:04,105 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,106 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,106 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,107 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,107 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,107 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,107 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:04,108 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7da52dc0[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:04,108 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:04,108 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,109 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:04,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:04,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:04,109 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,110 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,111 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,111 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,114 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:04,114 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:04,114 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:04,114 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:04,114 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:04,114 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:04,114 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,119 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,119 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,120 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,120 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,120 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,120 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,121 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,122 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,122 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,124 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:04,124 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,123 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,124 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,124 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,125 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,224 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:04,226 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:04,242 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:04,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:04,244 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:04,244 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:04,244 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,244 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:04,244 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:04,245 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,245 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,246 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,246 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,246 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:04,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,248 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,248 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,249 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,248 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,249 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:04,249 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:04,249 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,249 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:04,250 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,250 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,251 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,251 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,261 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 2023-04-08 22:05:04,275 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/44cbbf7254744bc9b556c670e9034b01 2023-04-08 22:05:04,279 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 2023-04-08 22:05:04,283 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:05:04,284 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/44cbbf7254744bc9b556c670e9034b01 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01 2023-04-08 22:05:04,288 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01, entries=10, sequenceid=58, filesize=5.0 K 2023-04-08 22:05:04,289 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 39ms, sequenceid=58, compaction requested=true 2023-04-08 22:05:04,289 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:04,289 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,289 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:04,289 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:04,289 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,289 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,289 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,291 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,291 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,291 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,291 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,293 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:04,294 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31432783[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:04,294 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:04,294 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,295 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,295 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:04,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:04,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,296 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:04,295 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,296 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:04,296 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,296 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,296 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,296 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,297 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:04,301 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:04,301 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:04,301 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:04,301 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:04,301 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:04,301 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,303 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,303 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,304 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,304 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,305 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,306 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,310 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,311 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,311 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,319 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,319 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,319 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:04,320 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:04,319 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,319 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,320 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,320 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,321 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,321 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,420 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:04,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:04,432 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(933): Client=jenkins//148.251.75.209 procedure request for: flush-table-proc 2023-04-08 22:05:04,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-04-08 22:05:04,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-04-08 22:05:04,433 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-04-08 22:05:04,433 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,433 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-04-08 22:05:04,433 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-04-08 22:05:04,434 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,434 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,434 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,435 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,435 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,434 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,435 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,435 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-04-08 22:05:04,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,436 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,436 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,437 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,43551,1680991397921' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,437 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41749,1680991397967' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-04-08 22:05:04,437 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,437 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-04-08 22:05:04,438 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. started... 2023-04-08 22:05:04,438 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-04-08 22:05:04,439 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,440 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,440 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,440 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,440 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,440 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,450 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 2023-04-08 22:05:04,464 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/6a49f3c4643d4a7ca10127a1eda53630 2023-04-08 22:05:04,467 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 2023-04-08 22:05:04,470 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:05:04,471 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/6a49f3c4643d4a7ca10127a1eda53630 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630 2023-04-08 22:05:04,474 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630, entries=10, sequenceid=71, filesize=5.0 K 2023-04-08 22:05:04,474 INFO [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 36ms, sequenceid=71, compaction requested=true 2023-04-08 22:05:04,474 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:04,475 DEBUG [rs(jenkins-hbase20.apache.org,41779,1680991398002)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,475 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-04-08 22:05:04,475 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-04-08 22:05:04,475 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,475 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-04-08 22:05:04,475 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-04-08 22:05:04,476 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,476 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,476 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,476 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase20.apache.org,41779,1680991398002' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-04-08 22:05:04,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@655cd746[Count = 0] remaining members to acquire global barrier 2023-04-08 22:05:04,477 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-04-08 22:05:04,477 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,478 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,478 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-04-08 22:05:04,478 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41749,1680991397967' in zk 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,41779,1680991398002' in zk 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-04-08 22:05:04,478 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase20.apache.org,43551,1680991397921' in zk 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41749,1680991397967' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-04-08 22:05:04,479 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,41779,1680991398002' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,479 DEBUG [member: 'jenkins-hbase20.apache.org,43551,1680991397921' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase20.apache.org,41779,1680991398002': 2023-04-08 22:05:04,482 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase20.apache.org,41779,1680991398002' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-04-08 22:05:04,482 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-04-08 22:05:04,482 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-04-08 22:05:04,482 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-04-08 22:05:04,482 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-04-08 22:05:04,482 INFO [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,482 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,482 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,483 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-04-08 22:05:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-04-08 22:05:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,484 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,485 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,485 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-04-08 22:05:04,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-04-08 22:05:04,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,487 DEBUG [(jenkins-hbase20.apache.org,39273,1680991396909)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-04-08 22:05:04,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegion because node does not exist (not an error) 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-04-08 22:05:04,487 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-04-08 22:05:04,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-04-08 22:05:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,487 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:04,488 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-04-08 22:05:04,488 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:04,488 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,488 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-04-08 22:05:04,488 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-04-08 22:05:04,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,488 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-04-08 22:05:04,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-04-08 22:05:04,588 DEBUG [Listener at localhost.localdomain/46567] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-04-08 22:05:04,589 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-04-08 22:05:04,604 DEBUG [Listener at localhost.localdomain/46567] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,605 DEBUG [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=75da9be8f486bf115207274ff1255b9f, daughterA=a0ed724c9e51270b685b5d8ef57d1912, daughterB=67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:04,606 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=75da9be8f486bf115207274ff1255b9f, daughterA=a0ed724c9e51270b685b5d8ef57d1912, daughterB=67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:04,606 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=75da9be8f486bf115207274ff1255b9f, daughterA=a0ed724c9e51270b685b5d8ef57d1912, daughterB=67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:04,606 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=75da9be8f486bf115207274ff1255b9f, daughterA=a0ed724c9e51270b685b5d8ef57d1912, daughterB=67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:04,609 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, UNASSIGN}] 2023-04-08 22:05:04,610 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, UNASSIGN 2023-04-08 22:05:04,611 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=75da9be8f486bf115207274ff1255b9f, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:04,611 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991504611"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991504611"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991504611"}]},"ts":"1680991504611"} 2023-04-08 22:05:04,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 75da9be8f486bf115207274ff1255b9f, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:04,765 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 75da9be8f486bf115207274ff1255b9f, disabling compactions & flushes 2023-04-08 22:05:04,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. after waiting 0 ms 2023-04-08 22:05:04,766 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,766 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 75da9be8f486bf115207274ff1255b9f 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-04-08 22:05:04,785 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/b1033c05efcf4d718832aa04149ae61e 2023-04-08 22:05:04,799 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/ceeb791e806e453eb38d4522885fb143 2023-04-08 22:05:04,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf1/b1033c05efcf4d718832aa04149ae61e as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e 2023-04-08 22:05:04,809 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:05:04,810 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/.tmp/cf2/ceeb791e806e453eb38d4522885fb143 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143 2023-04-08 22:05:04,815 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143, entries=8, sequenceid=82, filesize=5.0 K 2023-04-08 22:05:04,816 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 75da9be8f486bf115207274ff1255b9f in 50ms, sequenceid=82, compaction requested=true 2023-04-08 22:05:04,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-04-08 22:05:04,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:04,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. 2023-04-08 22:05:04,822 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 75da9be8f486bf115207274ff1255b9f: 2023-04-08 22:05:04,823 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,824 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=75da9be8f486bf115207274ff1255b9f, regionState=CLOSED 2023-04-08 22:05:04,824 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991504824"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991504824"}]},"ts":"1680991504824"} 2023-04-08 22:05:04,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-04-08 22:05:04,828 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 75da9be8f486bf115207274ff1255b9f, server=jenkins-hbase20.apache.org,41779,1680991398002 in 213 msec 2023-04-08 22:05:04,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-04-08 22:05:04,829 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=75da9be8f486bf115207274ff1255b9f, UNASSIGN in 219 msec 2023-04-08 22:05:04,843 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=75da9be8f486bf115207274ff1255b9f, threads=14 2023-04-08 22:05:04,844 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,844 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,844 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,844 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,844 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,845 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,845 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,845 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,845 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,845 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,846 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,846 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,846 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:04,846 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,068 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,069 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,071 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,071 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,071 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,073 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,083 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,084 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,085 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,085 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,086 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,086 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,088 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,088 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812 for region: 75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,088 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 75da9be8f486bf115207274ff1255b9f Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-04-08 22:05:05,119 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:05:05,121 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-04-08 22:05:05,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1680991505122"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1680991505122"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1680991505122"}]},"ts":"1680991505122"} 2023-04-08 22:05:05,122 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505122"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991505122"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991505122"}]},"ts":"1680991505122"} 2023-04-08 22:05:05,123 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505122"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991505122"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991505122"}]},"ts":"1680991505122"} 2023-04-08 22:05:05,126 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, ASSIGN}] 2023-04-08 22:05:05,127 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, ASSIGN 2023-04-08 22:05:05,128 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:05:05,128 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, ASSIGN 2023-04-08 22:05:05,129 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase20.apache.org,41779,1680991398002; forceNewPlan=false, retain=false 2023-04-08 22:05:05,278 INFO [jenkins-hbase20:39273] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-04-08 22:05:05,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=a0ed724c9e51270b685b5d8ef57d1912, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505281"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991505281"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991505281"}]},"ts":"1680991505281"} 2023-04-08 22:05:05,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure a0ed724c9e51270b685b5d8ef57d1912, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:05,288 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=67a9adc7ccad2cd21a54c1cd8bc7473c, regionState=OPENING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,288 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505288"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991505288"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991505288"}]},"ts":"1680991505288"} 2023-04-08 22:05:05,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 67a9adc7ccad2cd21a54c1cd8bc7473c, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:05,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,444 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 67a9adc7ccad2cd21a54c1cd8bc7473c, NAME => 'testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.', STARTKEY => '41', ENDKEY => ''} 2023-04-08 22:05:05,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:05,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,445 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:05,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,446 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,448 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,449 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1 2023-04-08 22:05:05,449 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1 2023-04-08 22:05:05,449 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 67a9adc7ccad2cd21a54c1cd8bc7473c columnFamilyName cf1 2023-04-08 22:05:05,461 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-top 2023-04-08 22:05:05,465 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-top 2023-04-08 22:05:05,468 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-top 2023-04-08 22:05:05,473 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-top 2023-04-08 22:05:05,478 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-top 2023-04-08 22:05:05,481 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-top 2023-04-08 22:05:05,484 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-top 2023-04-08 22:05:05,484 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(310): Store=67a9adc7ccad2cd21a54c1cd8bc7473c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:05,485 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,485 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2 2023-04-08 22:05:05,485 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2 2023-04-08 22:05:05,485 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 67a9adc7ccad2cd21a54c1cd8bc7473c columnFamilyName cf2 2023-04-08 22:05:05,495 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-top 2023-04-08 22:05:05,497 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-top 2023-04-08 22:05:05,500 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-top 2023-04-08 22:05:05,503 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-top 2023-04-08 22:05:05,505 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-top 2023-04-08 22:05:05,508 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-top 2023-04-08 22:05:05,511 DEBUG [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-top 2023-04-08 22:05:05,511 INFO [StoreOpener-67a9adc7ccad2cd21a54c1cd8bc7473c-1] regionserver.HStore(310): Store=67a9adc7ccad2cd21a54c1cd8bc7473c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:05,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,512 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,513 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:05,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened 67a9adc7ccad2cd21a54c1cd8bc7473c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65615114, jitterRate=-0.022258609533309937}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:05,514 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for 67a9adc7ccad2cd21a54c1cd8bc7473c: 2023-04-08 22:05:05,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c., pid=150, masterSystemTime=1680991505436 2023-04-08 22:05:05,515 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:05,516 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:05,516 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:05,516 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,516 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): 67a9adc7ccad2cd21a54c1cd8bc7473c/cf1 is initiating minor compaction (all files) 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 67a9adc7ccad2cd21a54c1cd8bc7473c/cf1 in testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,516 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): 67a9adc7ccad2cd21a54c1cd8bc7473c/cf2 is initiating minor compaction (all files) 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 67a9adc7ccad2cd21a54c1cd8bc7473c/cf2 in testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/.tmp, totalSize=34.9 K 2023-04-08 22:05:05,516 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-top] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/.tmp, totalSize=34.9 K 2023-04-08 22:05:05,517 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991503504 2023-04-08 22:05:05,517 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1680991503504 2023-04-08 22:05:05,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=67a9adc7ccad2cd21a54c1cd8bc7473c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,517 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a0ed724c9e51270b685b5d8ef57d1912, NAME => 'testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.', STARTKEY => '', ENDKEY => '41'} 2023-04-08 22:05:05,517 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505517"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991505517"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991505517"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991505517"}]},"ts":"1680991505517"} 2023-04-08 22:05:05,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-04-08 22:05:05,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-04-08 22:05:05,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7894): checking encryption for a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,518 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(7897): checking classloading for a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,518 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991503677 2023-04-08 22:05:05,518 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1680991503677 2023-04-08 22:05:05,519 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991503863 2023-04-08 22:05:05,519 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1680991503863 2023-04-08 22:05:05,519 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,519 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991504048 2023-04-08 22:05:05,519 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1680991504048 2023-04-08 22:05:05,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-04-08 22:05:05,519 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 67a9adc7ccad2cd21a54c1cd8bc7473c, server=jenkins-hbase20.apache.org,41779,1680991398002 in 228 msec 2023-04-08 22:05:05,520 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991504226 2023-04-08 22:05:05,520 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1 2023-04-08 22:05:05,520 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1 2023-04-08 22:05:05,520 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1680991504226 2023-04-08 22:05:05,520 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991504422 2023-04-08 22:05:05,520 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a0ed724c9e51270b685b5d8ef57d1912 columnFamilyName cf1 2023-04-08 22:05:05,520 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1680991504422 2023-04-08 22:05:05,520 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, ASSIGN in 393 msec 2023-04-08 22:05:05,520 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991504590 2023-04-08 22:05:05,521 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1680991504590 2023-04-08 22:05:05,528 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): 67a9adc7ccad2cd21a54c1cd8bc7473c#cf1#compaction#141 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:05,528 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): 67a9adc7ccad2cd21a54c1cd8bc7473c#cf2#compaction#140 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:05,531 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-bottom 2023-04-08 22:05:05,536 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-bottom 2023-04-08 22:05:05,538 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-bottom 2023-04-08 22:05:05,541 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-bottom 2023-04-08 22:05:05,544 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-bottom 2023-04-08 22:05:05,545 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/.tmp/cf2/e0a359eaea3a46868855874a191be099 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/e0a359eaea3a46868855874a191be099 2023-04-08 22:05:05,547 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/.tmp/cf1/4f4c31bfd18c4c64b0df7a43cd1170a9 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/4f4c31bfd18c4c64b0df7a43cd1170a9 2023-04-08 22:05:05,547 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-bottom 2023-04-08 22:05:05,548 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 67a9adc7ccad2cd21a54c1cd8bc7473c/cf2 of 67a9adc7ccad2cd21a54c1cd8bc7473c into e0a359eaea3a46868855874a191be099(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:05,548 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 67a9adc7ccad2cd21a54c1cd8bc7473c: 2023-04-08 22:05:05,548 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c., storeName=67a9adc7ccad2cd21a54c1cd8bc7473c/cf2, priority=9, startTime=1680991505516; duration=0sec 2023-04-08 22:05:05,548 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:05,550 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-bottom 2023-04-08 22:05:05,554 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(310): Store=a0ed724c9e51270b685b5d8ef57d1912/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:05,554 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,555 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2 2023-04-08 22:05:05,555 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2 2023-04-08 22:05:05,555 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 67a9adc7ccad2cd21a54c1cd8bc7473c/cf1 of 67a9adc7ccad2cd21a54c1cd8bc7473c into 4f4c31bfd18c4c64b0df7a43cd1170a9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:05,555 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 67a9adc7ccad2cd21a54c1cd8bc7473c: 2023-04-08 22:05:05,555 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c., storeName=67a9adc7ccad2cd21a54c1cd8bc7473c/cf1, priority=9, startTime=1680991505515; duration=0sec 2023-04-08 22:05:05,555 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a0ed724c9e51270b685b5d8ef57d1912 columnFamilyName cf2 2023-04-08 22:05:05,555 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:05,569 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-bottom 2023-04-08 22:05:05,572 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-bottom 2023-04-08 22:05:05,575 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-bottom 2023-04-08 22:05:05,577 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-bottom 2023-04-08 22:05:05,580 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-bottom 2023-04-08 22:05:05,583 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-bottom 2023-04-08 22:05:05,585 DEBUG [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-bottom 2023-04-08 22:05:05,586 INFO [StoreOpener-a0ed724c9e51270b685b5d8ef57d1912-1] regionserver.HStore(310): Store=a0ed724c9e51270b685b5d8ef57d1912/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-04-08 22:05:05,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-04-08 22:05:05,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1055): writing seq id for a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1072): Opened a0ed724c9e51270b685b5d8ef57d1912; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71106885, jitterRate=0.059575155377388}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-04-08 22:05:05,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(965): Region open journal for a0ed724c9e51270b685b5d8ef57d1912: 2023-04-08 22:05:05,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912., pid=149, masterSystemTime=1680991505436 2023-04-08 22:05:05,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:05:05,592 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:05,592 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-04-08 22:05:05,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-04-08 22:05:05,592 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,592 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,592 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1912): a0ed724c9e51270b685b5d8ef57d1912/cf1 is initiating minor compaction (all files) 2023-04-08 22:05:05,592 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1912): a0ed724c9e51270b685b5d8ef57d1912/cf2 is initiating minor compaction (all files) 2023-04-08 22:05:05,592 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2259): Starting compaction of a0ed724c9e51270b685b5d8ef57d1912/cf1 in testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,592 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of a0ed724c9e51270b685b5d8ef57d1912/cf2 in testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,593 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/.tmp, totalSize=34.9 K 2023-04-08 22:05:05,593 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-bottom] into tmpdir=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/.tmp, totalSize=34.9 K 2023-04-08 22:05:05,593 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,593 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991503504 2023-04-08 22:05:05,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase20:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,594 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=a0ed724c9e51270b685b5d8ef57d1912, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,594 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1680991503504 2023-04-08 22:05:05,594 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505593"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1680991505593"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1680991505593"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1680991505593"}]},"ts":"1680991505593"} 2023-04-08 22:05:05,594 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991503677 2023-04-08 22:05:05,594 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1680991503677 2023-04-08 22:05:05,594 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991503863 2023-04-08 22:05:05,595 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1680991503863 2023-04-08 22:05:05,595 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991504048 2023-04-08 22:05:05,595 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1680991504048 2023-04-08 22:05:05,595 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991504226 2023-04-08 22:05:05,595 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1680991504226 2023-04-08 22:05:05,596 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting 0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991504422 2023-04-08 22:05:05,596 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting 6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1680991504422 2023-04-08 22:05:05,596 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] compactions.Compactor(207): Compacting b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991504590 2023-04-08 22:05:05,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-04-08 22:05:05,596 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] compactions.Compactor(207): Compacting ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1680991504590 2023-04-08 22:05:05,596 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure a0ed724c9e51270b685b5d8ef57d1912, server=jenkins-hbase20.apache.org,41779,1680991398002 in 312 msec 2023-04-08 22:05:05,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-04-08 22:05:05,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, ASSIGN in 470 msec 2023-04-08 22:05:05,599 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=75da9be8f486bf115207274ff1255b9f, daughterA=a0ed724c9e51270b685b5d8ef57d1912, daughterB=67a9adc7ccad2cd21a54c1cd8bc7473c in 994 msec 2023-04-08 22:05:05,605 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] throttle.PressureAwareThroughputController(145): a0ed724c9e51270b685b5d8ef57d1912#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:05,606 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] throttle.PressureAwareThroughputController(145): a0ed724c9e51270b685b5d8ef57d1912#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-04-08 22:05:05,606 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=41779] ipc.CallRunner(144): callId: 1148 service: ClientService methodName: Scan size: 143 connection: 148.251.75.209:32990 deadline: 1680991565606, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f. is not online on jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,618 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/.tmp/cf1/39e03a95c37349d59e86d9a88d1aaa5c as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/39e03a95c37349d59e86d9a88d1aaa5c 2023-04-08 22:05:05,623 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a0ed724c9e51270b685b5d8ef57d1912/cf1 of a0ed724c9e51270b685b5d8ef57d1912 into 39e03a95c37349d59e86d9a88d1aaa5c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:05,623 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for a0ed724c9e51270b685b5d8ef57d1912: 2023-04-08 22:05:05,623 INFO [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912., storeName=a0ed724c9e51270b685b5d8ef57d1912/cf1, priority=9, startTime=1680991505591; duration=0sec 2023-04-08 22:05:05,623 DEBUG [RS:2;jenkins-hbase20:41779-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:05,627 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/.tmp/cf2/f10f883097bb4eaabb3f1f3df326bcfe as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/f10f883097bb4eaabb3f1f3df326bcfe 2023-04-08 22:05:05,630 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in a0ed724c9e51270b685b5d8ef57d1912/cf2 of a0ed724c9e51270b685b5d8ef57d1912 into f10f883097bb4eaabb3f1f3df326bcfe(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-04-08 22:05:05,630 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for a0ed724c9e51270b685b5d8ef57d1912: 2023-04-08 22:05:05,630 INFO [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912., storeName=a0ed724c9e51270b685b5d8ef57d1912/cf2, priority=9, startTime=1680991505592; duration=0sec 2023-04-08 22:05:05,630 DEBUG [RS:2;jenkins-hbase20:41779-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-04-08 22:05:05,655 ERROR [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-1236918431-148.251.75.209-1680991393780 (Datanode Uuid 77a94139-75e1-486c-a582-be7e9aa2826d) service to localhost.localdomain/127.0.0.1:34893 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-04-08 22:05:05,718 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:05:05,724 INFO [Listener at localhost.localdomain/46567] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-04-08 22:05:05,725 DEBUG [Listener at localhost.localdomain/46567] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-04-08 22:05:05,737 WARN [Listener at localhost.localdomain/46567] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-04-08 22:05:05,739 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-04-08 22:05:05,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$11(2418): Client=jenkins//148.251.75.209 disable testSplitTableRegion 2023-04-08 22:05:05,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-04-08 22:05:05,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-08 22:05:05,742 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991505742"}]},"ts":"1680991505742"} 2023-04-08 22:05:05,743 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-04-08 22:05:05,744 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-04-08 22:05:05,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, UNASSIGN}] 2023-04-08 22:05:05,745 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, UNASSIGN 2023-04-08 22:05:05,746 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=67a9adc7ccad2cd21a54c1cd8bc7473c, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,746 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505746"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991505746"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991505746"}]},"ts":"1680991505746"} 2023-04-08 22:05:05,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 67a9adc7ccad2cd21a54c1cd8bc7473c, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:05,748 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, UNASSIGN 2023-04-08 22:05:05,748 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=a0ed724c9e51270b685b5d8ef57d1912, regionState=CLOSING, regionLocation=jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:05,749 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505748"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1680991505748"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1680991505748"}]},"ts":"1680991505748"} 2023-04-08 22:05:05,751 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure a0ed724c9e51270b685b5d8ef57d1912, server=jenkins-hbase20.apache.org,41779,1680991398002}] 2023-04-08 22:05:05,844 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-08 22:05:05,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing a0ed724c9e51270b685b5d8ef57d1912, disabling compactions & flushes 2023-04-08 22:05:05,899 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,899 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. after waiting 0 ms 2023-04-08 22:05:05,900 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,901 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-bottom] to archive 2023-04-08 22:05:05,903 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:05,908 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,912 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,914 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,915 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,917 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,919 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,920 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,922 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-bottom, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-bottom] to archive 2023-04-08 22:05:05,923 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:05,925 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,926 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,928 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,930 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,932 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,934 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,936 DEBUG [StoreCloser-testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:05:05,941 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:05,942 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912. 2023-04-08 22:05:05,942 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for a0ed724c9e51270b685b5d8ef57d1912: 2023-04-08 22:05:05,943 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:05,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(109): Close 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 67a9adc7ccad2cd21a54c1cd8bc7473c, disabling compactions & flushes 2023-04-08 22:05:05,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. after waiting 0 ms 2023-04-08 22:05:05,944 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=a0ed724c9e51270b685b5d8ef57d1912, regionState=CLOSED 2023-04-08 22:05:05,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,944 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505944"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991505944"}]},"ts":"1680991505944"} 2023-04-08 22:05:05,946 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e-top] to archive 2023-04-08 22:05:05,947 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:05,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-04-08 22:05:05,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure a0ed724c9e51270b685b5d8ef57d1912, server=jenkins-hbase20.apache.org,41779,1680991398002 in 197 msec 2023-04-08 22:05:05,949 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/ec5498f0626a4c8b8e90d8f7aa046812.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=a0ed724c9e51270b685b5d8ef57d1912, UNASSIGN in 203 msec 2023-04-08 22:05:05,951 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/fb65bdfde2324d7087e3a54f965ef58e.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,953 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/056f010dd37c4bd196cc728e7e22f67d.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,954 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/7af2681fb86b47269cdbc00f13d5917b.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,955 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/402040e6f03a4ce9ba7e1bf3a6be6261.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,956 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/0a801ed3f8b042e59bc5b9e98e349eb2.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,957 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/b1033c05efcf4d718832aa04149ae61e.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,959 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630-top, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f->hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143-top] to archive 2023-04-08 22:05:05,960 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-04-08 22:05:05,961 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6cacb1af20c94c30a0cb9e8eacbb769b.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,962 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,964 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/bdcff42970c542f589885bdfac76b8fa.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,965 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/56e42bdc6c7845ad915268372feaa6b9.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,966 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/44cbbf7254744bc9b556c670e9034b01.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,967 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/6a49f3c4643d4a7ca10127a1eda53630.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,968 DEBUG [StoreCloser-testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/ceeb791e806e453eb38d4522885fb143.75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:05,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-04-08 22:05:05,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:05,971 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c. 2023-04-08 22:05:05,971 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 67a9adc7ccad2cd21a54c1cd8bc7473c: 2023-04-08 22:05:05,972 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.UnassignRegionHandler(147): Closed 67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:05,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=67a9adc7ccad2cd21a54c1cd8bc7473c, regionState=CLOSED 2023-04-08 22:05:05,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1680991505972"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1680991505972"}]},"ts":"1680991505972"} 2023-04-08 22:05:05,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-04-08 22:05:05,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 67a9adc7ccad2cd21a54c1cd8bc7473c, server=jenkins-hbase20.apache.org,41779,1680991398002 in 226 msec 2023-04-08 22:05:05,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-04-08 22:05:05,975 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=67a9adc7ccad2cd21a54c1cd8bc7473c, UNASSIGN in 229 msec 2023-04-08 22:05:05,976 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1680991505976"}]},"ts":"1680991505976"} 2023-04-08 22:05:05,977 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-04-08 22:05:05,977 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-04-08 22:05:05,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 239 msec 2023-04-08 22:05:06,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-04-08 22:05:06,046 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-04-08 22:05:06,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.HMaster$5(2228): Client=jenkins//148.251.75.209 delete testSplitTableRegion 2023-04-08 22:05:06,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,049 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,049 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-04-08 22:05:06,053 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:06,053 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:06,053 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:06,054 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/recovered.edits] 2023-04-08 22:05:06,054 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/recovered.edits] 2023-04-08 22:05:06,054 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2, FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/recovered.edits] 2023-04-08 22:05:06,057 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/4f4c31bfd18c4c64b0df7a43cd1170a9 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf1/4f4c31bfd18c4c64b0df7a43cd1170a9 2023-04-08 22:05:06,057 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/39e03a95c37349d59e86d9a88d1aaa5c to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf1/39e03a95c37349d59e86d9a88d1aaa5c 2023-04-08 22:05:06,057 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/056f010dd37c4bd196cc728e7e22f67d 2023-04-08 22:05:06,059 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/0a801ed3f8b042e59bc5b9e98e349eb2 2023-04-08 22:05:06,059 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/e0a359eaea3a46868855874a191be099 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/cf2/e0a359eaea3a46868855874a191be099 2023-04-08 22:05:06,059 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/f10f883097bb4eaabb3f1f3df326bcfe to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/cf2/f10f883097bb4eaabb3f1f3df326bcfe 2023-04-08 22:05:06,060 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/402040e6f03a4ce9ba7e1bf3a6be6261 2023-04-08 22:05:06,061 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/7af2681fb86b47269cdbc00f13d5917b 2023-04-08 22:05:06,061 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912/recovered.edits/90.seqid 2023-04-08 22:05:06,061 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/recovered.edits/90.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c/recovered.edits/90.seqid 2023-04-08 22:05:06,062 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/b1033c05efcf4d718832aa04149ae61e 2023-04-08 22:05:06,062 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/67a9adc7ccad2cd21a54c1cd8bc7473c 2023-04-08 22:05:06,062 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/a0ed724c9e51270b685b5d8ef57d1912 2023-04-08 22:05:06,063 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/ec5498f0626a4c8b8e90d8f7aa046812 2023-04-08 22:05:06,064 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf1/fb65bdfde2324d7087e3a54f965ef58e 2023-04-08 22:05:06,065 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/3cb72a5fc0bd4ec3b85636c9b1c2c0a3 2023-04-08 22:05:06,066 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/44cbbf7254744bc9b556c670e9034b01 2023-04-08 22:05:06,067 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/56e42bdc6c7845ad915268372feaa6b9 2023-04-08 22:05:06,068 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6a49f3c4643d4a7ca10127a1eda53630 2023-04-08 22:05:06,068 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/6cacb1af20c94c30a0cb9e8eacbb769b 2023-04-08 22:05:06,069 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/bdcff42970c542f589885bdfac76b8fa 2023-04-08 22:05:06,070 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143 to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/cf2/ceeb791e806e453eb38d4522885fb143 2023-04-08 22:05:06,071 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/recovered.edits/85.seqid to hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/archive/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f/recovered.edits/85.seqid 2023-04-08 22:05:06,071 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/.tmp/data/default/testSplitTableRegion/75da9be8f486bf115207274ff1255b9f 2023-04-08 22:05:06,071 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-04-08 22:05:06,073 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,074 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-04-08 22:05:06,076 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-04-08 22:05:06,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-04-08 22:05:06,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991506077"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:06,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991506077"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:06,077 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1680991506077"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:06,079 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-04-08 22:05:06,079 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 75da9be8f486bf115207274ff1255b9f, NAME => 'testSplitTableRegion,,1680991502998.75da9be8f486bf115207274ff1255b9f.', STARTKEY => '', ENDKEY => ''}, {ENCODED => a0ed724c9e51270b685b5d8ef57d1912, NAME => 'testSplitTableRegion,,1680991504604.a0ed724c9e51270b685b5d8ef57d1912.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 67a9adc7ccad2cd21a54c1cd8bc7473c, NAME => 'testSplitTableRegion,41,1680991504604.67a9adc7ccad2cd21a54c1cd8bc7473c.', STARTKEY => '41', ENDKEY => ''}] 2023-04-08 22:05:06,079 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-04-08 22:05:06,079 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1680991506079"}]},"ts":"9223372036854775807"} 2023-04-08 22:05:06,080 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-04-08 22:05:06,081 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-04-08 22:05:06,082 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 35 msec 2023-04-08 22:05:06,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=39273] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-04-08 22:05:06,151 INFO [Listener at localhost.localdomain/46567] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-04-08 22:05:06,173 INFO [Listener at localhost.localdomain/46567] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 377) - Thread LEAK? -, OpenFileDescriptor=833 (was 833), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=95 (was 95), ProcessCount=162 (was 162), AvailableMemoryMB=4752 (was 4834) 2023-04-08 22:05:06,176 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-04-08 22:05:06,178 INFO [Listener at localhost.localdomain/46567] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-04-08 22:05:06,179 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1afd0182 to 127.0.0.1:55350 2023-04-08 22:05:06,179 DEBUG [Listener at localhost.localdomain/46567] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,180 DEBUG [Listener at localhost.localdomain/46567] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-04-08 22:05:06,180 DEBUG [Listener at localhost.localdomain/46567] util.JVMClusterUtil(257): Found active master hash=1686018663, stopped=false 2023-04-08 22:05:06,180 INFO [Listener at localhost.localdomain/46567] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-08 22:05:06,181 INFO [Listener at localhost.localdomain/46567] procedure2.ProcedureExecutor(629): Stopping 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-04-08 22:05:06,181 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:05:06,181 DEBUG [Listener at localhost.localdomain/46567] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38785559 to 127.0.0.1:55350 2023-04-08 22:05:06,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:05:06,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:05:06,182 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-04-08 22:05:06,182 DEBUG [Listener at localhost.localdomain/46567] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,182 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,43551,1680991397921' ***** 2023-04-08 22:05:06,182 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-08 22:05:06,182 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,41749,1680991397967' ***** 2023-04-08 22:05:06,182 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-08 22:05:06,182 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,41779,1680991398002' ***** 2023-04-08 22:05:06,183 INFO [Listener at localhost.localdomain/46567] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-04-08 22:05:06,183 INFO [RS:0;jenkins-hbase20:43551] regionserver.HeapMemoryManager(220): Stopping 2023-04-08 22:05:06,183 INFO [RS:1;jenkins-hbase20:41749] regionserver.HeapMemoryManager(220): Stopping 2023-04-08 22:05:06,183 INFO [RS:1;jenkins-hbase20:41749] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-08 22:05:06,183 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-08 22:05:06,183 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-08 22:05:06,183 INFO [RS:2;jenkins-hbase20:41779] regionserver.HeapMemoryManager(220): Stopping 2023-04-08 22:05:06,183 INFO [RS:0;jenkins-hbase20:43551] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-08 22:05:06,183 INFO [RS:2;jenkins-hbase20:41779] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-04-08 22:05:06,183 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-04-08 22:05:06,183 INFO [RS:1;jenkins-hbase20:41749] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-08 22:05:06,184 INFO [RS:2;jenkins-hbase20:41779] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-08 22:05:06,183 INFO [RS:0;jenkins-hbase20:43551] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-04-08 22:05:06,184 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:06,184 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:06,184 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(3304): Received CLOSE for b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:05:06,184 DEBUG [RS:0;jenkins-hbase20:43551] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x3c7f3513 to 127.0.0.1:55350 2023-04-08 22:05:06,184 DEBUG [RS:2;jenkins-hbase20:41779] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28fb1460 to 127.0.0.1:55350 2023-04-08 22:05:06,184 DEBUG [RS:0;jenkins-hbase20:43551] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,184 DEBUG [RS:2;jenkins-hbase20:41779] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,184 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,43551,1680991397921; all regions closed. 2023-04-08 22:05:06,184 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:06,184 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,41779,1680991398002; all regions closed. 2023-04-08 22:05:06,184 DEBUG [RS:1;jenkins-hbase20:41749] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x23e3cc80 to 127.0.0.1:55350 2023-04-08 22:05:06,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing b6167fccb8f7fec5f3eb188ccc6fd56e, disabling compactions & flushes 2023-04-08 22:05:06,184 DEBUG [RS:1;jenkins-hbase20:41749] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:05:06,185 INFO [RS:1;jenkins-hbase20:41749] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:05:06,185 INFO [RS:1;jenkins-hbase20:41749] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. after waiting 0 ms 2023-04-08 22:05:06,185 INFO [RS:1;jenkins-hbase20:41749] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:05:06,185 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-04-08 22:05:06,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing b6167fccb8f7fec5f3eb188ccc6fd56e 1/1 column families, dataSize=78 B heapSize=488 B 2023-04-08 22:05:06,185 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1475): Waiting on 2 regions to close 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-04-08 22:05:06,185 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-04-08 22:05:06,185 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740, b6167fccb8f7fec5f3eb188ccc6fd56e=hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e.} 2023-04-08 22:05:06,185 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-04-08 22:05:06,186 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-04-08 22:05:06,186 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-04-08 22:05:06,186 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1505): Waiting on 1588230740, b6167fccb8f7fec5f3eb188ccc6fd56e 2023-04-08 22:05:06,204 DEBUG [RS:2;jenkins-hbase20:41779] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs 2023-04-08 22:05:06,205 INFO [RS:2;jenkins-hbase20:41779] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C41779%2C1680991398002:(num 1680991399481) 2023-04-08 22:05:06,205 DEBUG [RS:2;jenkins-hbase20:41779] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,205 INFO [RS:2;jenkins-hbase20:41779] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,215 INFO [RS:2;jenkins-hbase20:41779] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-08 22:05:06,215 INFO [RS:2;jenkins-hbase20:41779] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-08 22:05:06,215 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-08 22:05:06,215 INFO [RS:2;jenkins-hbase20:41779] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-08 22:05:06,216 INFO [RS:2;jenkins-hbase20:41779] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-08 22:05:06,217 INFO [RS:2;jenkins-hbase20:41779] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:41779 2023-04-08 22:05:06,223 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/.tmp/info/1721bc2ddfa54bc2a74730f03f31efee 2023-04-08 22:05:06,224 DEBUG [RS:0;jenkins-hbase20:43551] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs 2023-04-08 22:05:06,224 INFO [RS:0;jenkins-hbase20:43551] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C43551%2C1680991397921:(num 1680991399481) 2023-04-08 22:05:06,224 DEBUG [RS:0;jenkins-hbase20:43551] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,224 INFO [RS:0;jenkins-hbase20:43551] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,224 INFO [RS:0;jenkins-hbase20:43551] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-08 22:05:06,224 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:06,224 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:06,225 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,225 INFO [RS:0;jenkins-hbase20:43551] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-04-08 22:05:06,225 INFO [RS:0;jenkins-hbase20:43551] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-04-08 22:05:06,225 INFO [RS:0;jenkins-hbase20:43551] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-04-08 22:05:06,225 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-08 22:05:06,225 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,225 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,225 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41779,1680991398002 2023-04-08 22:05:06,226 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@22f6630c rejected from java.util.concurrent.ThreadPoolExecutor@26c9f38a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,226 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,227 INFO [RS:0;jenkins-hbase20:43551] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:43551 2023-04-08 22:05:06,227 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@439dcbfc rejected from java.util.concurrent.ThreadPoolExecutor@26c9f38a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,227 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,227 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,227 INFO [regionserver/jenkins-hbase20:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,227 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,41779,1680991398002] 2023-04-08 22:05:06,227 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,41779,1680991398002; numProcessing=1 2023-04-08 22:05:06,228 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:06,229 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,229 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,43551,1680991397921 2023-04-08 22:05:06,229 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1c2aebde rejected from java.util.concurrent.ThreadPoolExecutor@18157c83[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,230 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,41779,1680991398002 already deleted, retry=false 2023-04-08 22:05:06,230 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,41779,1680991398002 expired; onlineServers=2 2023-04-08 22:05:06,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/.tmp/info/1721bc2ddfa54bc2a74730f03f31efee as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/info/1721bc2ddfa54bc2a74730f03f31efee 2023-04-08 22:05:06,231 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,43551,1680991397921] 2023-04-08 22:05:06,231 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,43551,1680991397921; numProcessing=2 2023-04-08 22:05:06,232 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,43551,1680991397921 already deleted, retry=false 2023-04-08 22:05:06,232 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,43551,1680991397921 expired; onlineServers=1 2023-04-08 22:05:06,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/info/1721bc2ddfa54bc2a74730f03f31efee, entries=2, sequenceid=6, filesize=4.8 K 2023-04-08 22:05:06,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for b6167fccb8f7fec5f3eb188ccc6fd56e in 50ms, sequenceid=6, compaction requested=false 2023-04-08 22:05:06,239 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/namespace/b6167fccb8f7fec5f3eb188ccc6fd56e/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-04-08 22:05:06,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:06,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:05:06,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for b6167fccb8f7fec5f3eb188ccc6fd56e: 2023-04-08 22:05:06,240 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1680991399894.b6167fccb8f7fec5f3eb188ccc6fd56e. 2023-04-08 22:05:06,320 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-04-08 22:05:06,320 INFO [regionserver/jenkins-hbase20:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-04-08 22:05:06,328 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,328 INFO [RS:2;jenkins-hbase20:41779] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,41779,1680991398002; zookeeper connection closed. 2023-04-08 22:05:06,328 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3bded2e5 rejected from java.util.concurrent.ThreadPoolExecutor@26c9f38a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,328 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41779-0x100929b39380003, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,328 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@158d4e9f] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@158d4e9f 2023-04-08 22:05:06,328 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@2e8b7fd3 rejected from java.util.concurrent.ThreadPoolExecutor@26c9f38a[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,386 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-08 22:05:06,428 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,428 INFO [RS:0;jenkins-hbase20:43551] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,43551,1680991397921; zookeeper connection closed. 2023-04-08 22:05:06,428 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@273f273b rejected from java.util.concurrent.ThreadPoolExecutor@18157c83[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,429 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@455d6e32] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@455d6e32 2023-04-08 22:05:06,429 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:43551-0x100929b39380001, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,429 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5f42c46e rejected from java.util.concurrent.ThreadPoolExecutor@18157c83[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,587 DEBUG [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-04-08 22:05:06,620 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/info/15f2dc3efae84fcca079cff1916dd81a 2023-04-08 22:05:06,627 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15f2dc3efae84fcca079cff1916dd81a 2023-04-08 22:05:06,638 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/rep_barrier/e187bcf95943415cb269f7c3fdd421c7 2023-04-08 22:05:06,643 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e187bcf95943415cb269f7c3fdd421c7 2023-04-08 22:05:06,656 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/table/ca43d35dd4cd4fb8910bb53646cddf64 2023-04-08 22:05:06,659 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca43d35dd4cd4fb8910bb53646cddf64 2023-04-08 22:05:06,659 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/info/15f2dc3efae84fcca079cff1916dd81a as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/info/15f2dc3efae84fcca079cff1916dd81a 2023-04-08 22:05:06,663 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 15f2dc3efae84fcca079cff1916dd81a 2023-04-08 22:05:06,663 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/info/15f2dc3efae84fcca079cff1916dd81a, entries=59, sequenceid=210, filesize=11.1 K 2023-04-08 22:05:06,663 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/rep_barrier/e187bcf95943415cb269f7c3fdd421c7 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/rep_barrier/e187bcf95943415cb269f7c3fdd421c7 2023-04-08 22:05:06,666 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e187bcf95943415cb269f7c3fdd421c7 2023-04-08 22:05:06,666 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/rep_barrier/e187bcf95943415cb269f7c3fdd421c7, entries=24, sequenceid=210, filesize=7.5 K 2023-04-08 22:05:06,667 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/.tmp/table/ca43d35dd4cd4fb8910bb53646cddf64 as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/table/ca43d35dd4cd4fb8910bb53646cddf64 2023-04-08 22:05:06,670 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for ca43d35dd4cd4fb8910bb53646cddf64 2023-04-08 22:05:06,670 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/table/ca43d35dd4cd4fb8910bb53646cddf64, entries=36, sequenceid=210, filesize=8.0 K 2023-04-08 22:05:06,671 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 485ms, sequenceid=210, compaction requested=false 2023-04-08 22:05:06,675 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-04-08 22:05:06,675 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:06,675 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-08 22:05:06,676 INFO [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-04-08 22:05:06,676 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-04-08 22:05:06,676 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase20:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-04-08 22:05:06,787 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,41749,1680991397967; all regions closed. 2023-04-08 22:05:06,798 DEBUG [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs 2023-04-08 22:05:06,799 INFO [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C41749%2C1680991397967.meta:.meta(num 1680991399622) 2023-04-08 22:05:06,803 DEBUG [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/oldWALs 2023-04-08 22:05:06,803 INFO [RS:1;jenkins-hbase20:41749] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase20.apache.org%2C41749%2C1680991397967:(num 1680991399481) 2023-04-08 22:05:06,803 DEBUG [RS:1;jenkins-hbase20:41749] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,803 INFO [RS:1;jenkins-hbase20:41749] regionserver.LeaseManager(133): Closed leases 2023-04-08 22:05:06,803 INFO [RS:1;jenkins-hbase20:41749] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase20:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-04-08 22:05:06,804 INFO [regionserver/jenkins-hbase20:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-08 22:05:06,804 INFO [RS:1;jenkins-hbase20:41749] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:41749 2023-04-08 22:05:06,805 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase20.apache.org,41749,1680991397967 2023-04-08 22:05:06,805 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-04-08 22:05:06,805 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@170b245e rejected from java.util.concurrent.ThreadPoolExecutor@1334b267[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,806 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase20.apache.org,41749,1680991397967] 2023-04-08 22:05:06,806 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase20.apache.org,41749,1680991397967; numProcessing=3 2023-04-08 22:05:06,806 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase20.apache.org,41749,1680991397967 already deleted, retry=false 2023-04-08 22:05:06,806 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase20.apache.org,41749,1680991397967 expired; onlineServers=0 2023-04-08 22:05:06,806 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase20.apache.org,39273,1680991396909' ***** 2023-04-08 22:05:06,806 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-04-08 22:05:06,807 DEBUG [M:0;jenkins-hbase20:39273] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2543905c, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase20.apache.org/148.251.75.209:0 2023-04-08 22:05:06,807 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegionServer(1145): stopping server jenkins-hbase20.apache.org,39273,1680991396909 2023-04-08 22:05:06,807 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegionServer(1171): stopping server jenkins-hbase20.apache.org,39273,1680991396909; all regions closed. 2023-04-08 22:05:06,807 DEBUG [M:0;jenkins-hbase20:39273] ipc.AbstractRpcClient(494): Stopping rpc client 2023-04-08 22:05:06,807 DEBUG [M:0;jenkins-hbase20:39273] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-04-08 22:05:06,807 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-04-08 22:05:06,807 DEBUG [M:0;jenkins-hbase20:39273] cleaner.HFileCleaner(317): Stopping file delete threads 2023-04-08 22:05:06,807 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1680991399122] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.small.0-1680991399122,5,FailOnTimeoutGroup] 2023-04-08 22:05:06,807 DEBUG [master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1680991399121] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase20:0:becomeActiveMaster-HFileCleaner.large.0-1680991399121,5,FailOnTimeoutGroup] 2023-04-08 22:05:06,808 INFO [M:0;jenkins-hbase20:39273] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-04-08 22:05:06,808 INFO [M:0;jenkins-hbase20:39273] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-04-08 22:05:06,808 INFO [M:0;jenkins-hbase20:39273] hbase.ChoreService(369): Chore service for: master/jenkins-hbase20:0 had [] on shutdown 2023-04-08 22:05:06,808 DEBUG [M:0;jenkins-hbase20:39273] master.HMaster(1512): Stopping service threads 2023-04-08 22:05:06,808 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-04-08 22:05:06,808 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-04-08 22:05:06,808 INFO [M:0;jenkins-hbase20:39273] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-04-08 22:05:06,808 ERROR [M:0;jenkins-hbase20:39273] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-04-08 22:05:06,808 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-04-08 22:05:06,808 INFO [M:0;jenkins-hbase20:39273] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-04-08 22:05:06,808 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-04-08 22:05:06,809 DEBUG [M:0;jenkins-hbase20:39273] zookeeper.ZKUtil(398): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-04-08 22:05:06,809 WARN [M:0;jenkins-hbase20:39273] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-04-08 22:05:06,809 INFO [M:0;jenkins-hbase20:39273] assignment.AssignmentManager(315): Stopping assignment manager 2023-04-08 22:05:06,809 INFO [M:0;jenkins-hbase20:39273] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-04-08 22:05:06,809 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-04-08 22:05:06,809 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:05:06,809 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:05:06,809 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-04-08 22:05:06,809 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:05:06,809 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.79 KB heapSize=719.71 KB 2023-04-08 22:05:06,818 INFO [M:0;jenkins-hbase20:39273] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.79 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/290ff8e06a8a46519b16a6209d2bcf3d 2023-04-08 22:05:06,821 INFO [M:0;jenkins-hbase20:39273] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 290ff8e06a8a46519b16a6209d2bcf3d 2023-04-08 22:05:06,821 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/290ff8e06a8a46519b16a6209d2bcf3d as hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/290ff8e06a8a46519b16a6209d2bcf3d 2023-04-08 22:05:06,824 INFO [M:0;jenkins-hbase20:39273] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 290ff8e06a8a46519b16a6209d2bcf3d 2023-04-08 22:05:06,825 INFO [M:0;jenkins-hbase20:39273] regionserver.HStore(1080): Added hdfs://localhost.localdomain:34893/user/jenkins/test-data/a5650446-4c22-921e-adf4-e19aa26cbff7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/290ff8e06a8a46519b16a6209d2bcf3d, entries=156, sequenceid=1282, filesize=34.9 K 2023-04-08 22:05:06,825 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegion(2948): Finished flush of dataSize ~608.79 KB/623403, heapSize ~719.70 KB/736968, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 16ms, sequenceid=1282, compaction requested=false 2023-04-08 22:05:06,826 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-04-08 22:05:06,826 DEBUG [M:0;jenkins-hbase20:39273] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-04-08 22:05:06,828 INFO [M:0;jenkins-hbase20:39273] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-04-08 22:05:06,828 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-04-08 22:05:06,829 INFO [M:0;jenkins-hbase20:39273] ipc.NettyRpcServer(158): Stopping server on /148.251.75.209:39273 2023-04-08 22:05:06,830 DEBUG [M:0;jenkins-hbase20:39273] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase20.apache.org,39273,1680991396909 already deleted, retry=false 2023-04-08 22:05:06,906 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,906 INFO [RS:1;jenkins-hbase20:41749] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,41749,1680991397967; zookeeper connection closed. 2023-04-08 22:05:06,906 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@294d51d5 rejected from java.util.concurrent.ThreadPoolExecutor@1334b267[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,906 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@551badf] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@551badf 2023-04-08 22:05:06,906 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): regionserver:41749-0x100929b39380002, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:06,907 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@513b986e rejected from java.util.concurrent.ThreadPoolExecutor@1334b267[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:06,907 INFO [Listener at localhost.localdomain/46567] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-04-08 22:05:07,006 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:07,006 INFO [M:0;jenkins-hbase20:39273] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase20.apache.org,39273,1680991396909; zookeeper connection closed. 2023-04-08 22:05:07,007 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3e2dda80 rejected from java.util.concurrent.ThreadPoolExecutor@26738643[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:07,007 DEBUG [Listener at localhost.localdomain/46567-EventThread] zookeeper.ZKWatcher(600): master:39273-0x100929b39380000, quorum=127.0.0.1:55350, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-04-08 22:05:07,007 ERROR [Listener at localhost.localdomain/46567-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@57a53a17 rejected from java.util.concurrent.ThreadPoolExecutor@26738643[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 753] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-04-08 22:05:07,010 WARN [Listener at localhost.localdomain/46567] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-08 22:05:07,016 INFO [Listener at localhost.localdomain/46567] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-08 22:05:07,128 WARN [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-04-08 22:05:07,128 WARN [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1236918431-148.251.75.209-1680991393780 (Datanode Uuid 0f83e86d-490c-4535-bc1b-a2692e207dbe) service to localhost.localdomain/127.0.0.1:34893 2023-04-08 22:05:07,131 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data5/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,132 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data6/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,133 WARN [Listener at localhost.localdomain/46567] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-08 22:05:07,135 INFO [Listener at localhost.localdomain/46567] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-08 22:05:07,236 WARN [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1236918431-148.251.75.209-1680991393780 (Datanode Uuid 77a94139-75e1-486c-a582-be7e9aa2826d) service to localhost.localdomain/127.0.0.1:34893 2023-04-08 22:05:07,238 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data3/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,239 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data4/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,239 WARN [Listener at localhost.localdomain/46567] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-04-08 22:05:07,243 INFO [Listener at localhost.localdomain/46567] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-04-08 22:05:07,355 WARN [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-04-08 22:05:07,355 WARN [BP-1236918431-148.251.75.209-1680991393780 heartbeating to localhost.localdomain/127.0.0.1:34893] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-1236918431-148.251.75.209-1680991393780 (Datanode Uuid da4aac5e-824a-49d2-8862-fa92d980a6ea) service to localhost.localdomain/127.0.0.1:34893 2023-04-08 22:05:07,357 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data1/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,357 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/28c560ac-85b2-63e7-28b8-96bbd5230620/cluster_95779944-d3e1-c588-ec13-e6988aca5cc1/dfs/data/data2/current/BP-1236918431-148.251.75.209-1680991393780] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-04-08 22:05:07,382 INFO [Listener at localhost.localdomain/46567] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-04-08 22:05:07,388 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-04-08 22:05:07,419 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-04-08 22:05:07,419 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(152): Removing adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-04-08 22:05:07,556 INFO [Listener at localhost.localdomain/46567] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-04-08 22:05:07,773 INFO [Listener at localhost.localdomain/46567] hbase.HBaseTestingUtility(1293): Minicluster is down